[Short Tip] Use Red Hat Satellite 6 as an inventory resource in Ansible

Ansible Logo

Besides static file inventories, Ansible can use custom scripts to dynamically generate inventories or access other sources, for example a CMDB or a system management server – like Red Hat Satellite.
Luckily, Nick Strugnell has already written a custom script to use Satellite as an inventory source in Ansible.

After checking out the git, the hammer.ini needs to be adjusted: at least host, username, password and organization must be adjusted.

Afterwards, the script can be invoked directly to show the available hosts:

$ ansible -i ~/Github/ansible-satellite6/satellite-inventory.py all --list-hosts
    argon.example.com
    satellite-server.example.com
    helium.example.com
...

This works with ansible CLI and playbook calls:

$ ansible-playbook -i ~/Github/ansible-satellite6/satellite-inventory.py apache-setup.yml
PLAY [apache setup] *********************************************************** 

GATHERING FACTS *************************************************************** 
...

The script works quite well – as long as the certificate you use on the Satellite server is trusted. Otherwise the value for self.ssl_verify must be set to False. Besides, it is a nice and simple way to access already existing inventory stores. This is important because Ansible is all about integration, and not about “throwing away and making new”.

[Howto] Look up of external sources in Ansible

Ansible Logo Part of Ansible’s power comes from an easy integration with other systems. In this post I will cover how to look up data from external sources like DNS or Redis.

Background

A tool for automation is only as good as it is capable to integrate it with the already existing environment – thus with other tools. Among various ways Ansible offers the possibility to look up Ansible variables from external stores like DNS, Redis, etcd or even generic INI or CSV files. This enables Ansible to easily access data which are stored – and changed, managed – outside of Ansible.

Setup

Ansible’s lookup feature is already installed by default.

Queries are executed on the host where the playbook is executed – in case of Tower this would be the Tower host itself. Thus the node needs access to the resources which needs to be queried.

Some lookup functions for example for DNS or Redis servers require additional python libraries – on the host actually executing the queries! On Fedora, the python-dns package is necessary for DNS queries and the package python-redis for Redis queries.

Generic usage

The lookup function can be used the exact same way variables are used: curly brackets surround the lookup function, the result is placed where the variable would be. That means lookup functions can be used in the head of a playbook, inside the tasks, even in templates.

The lookup command itself has to list the plugin as well as the arguments for the plugin:

{{ lookup('plugin','arguments') }}

Examples

Files

Entire files can be used as content of a variable. This is simply done via:

vars:
  content: "{{ lookup('file','lorem.txt') }}"

As a result, the variable has the entire content of the file. Note that the lookup of files always searches the files relative to the path of the actual playbook, not relative to the path where the command is executed.

Also, the lookup might fail when the file itself contains quote characters.

CVS

While the file lookup is pretty simple and generic, the CVS lookup module gives the ability to access values of given keys in a CVS file. An optional parameter can identify the appropriate column. For example, if the following CSV file is given:

$ cat gamma.csv
daytime,time,meal
breakfast,7,soup
lunch,12,rice
tea,15,cake
dinner,18,noodles

Now the lookup function for CVS files can access the lines identified by keys which are compared to the values of the first column. The following example looks up the key dinner and gives back the entry of the third column: {{ lookup('csvfile','dinner file=gamma.csv delimiter=, col=2') }}.

Inserted in a playbook, this looks like:

ansible-playbook examples/lookup.yml

PLAY [demo lookups] *********************************************************** 

GATHERING FACTS *************************************************************** 
ok: [neon]

TASK: [lookup of a cvs file] ************************************************** 
ok: [neon] => {
    "msg": "noodles"
}

PLAY RECAP ******************************************************************** 
neon                       : ok=2    changed=0    unreachable=0    failed=0

The corresponding playbook gives out the variable via the debug module:

---
- name: demo lookups
  hosts: neon

  tasks:
    - name: lookup of a cvs file
      debug: msg="{{ lookup('csvfile','dinner file=gamma.csv delimiter=, col=2') }}"

DNS

The DNS lookup is particularly interesting in cases where the local DNS provides a lot of information like SSH fingerprints or the MX record.

The DNS lookup plugin is called dig – like the command line client dig. As arguments, the plugin takes a domain name and the DNS type: {{ lookup('dig', 'redhat.com. qtype=MX') }}. Another way to hand over the type argument is via slash: {{ lookup('dig', 'redhat.com./MX') }}

The result for this example is:

TASK: [lookup of dns dig entries] ********************************************* 
ok: [neon] => {
    "msg": "10 int-mx.corp.redhat.com."
}

Redis

It gets even more interesting when existing databases are queried. Ansible lookup supports for example Redis databases. The plugin takes as argument the entire URL: redis://$URL:$PORT,$KEY.

For example, to query a local Redis server for the key dinner:

---
tasks:
  - name: lookup of redis entries
    debug: msg="{{ lookup('redis_kv', 'redis://localhost:6379,dinner') }}" 

The result is:

TASK: [lookup of redis entries] *********************************************** 
ok: [neon] => {
    "msg": "noodles"
}

Template

As already mentioned, lookups can not only be used in Playbooks, but also directly in templates. For example, given the template code:

$ cat templatej2
...
Red Hat MX: {{ lookup('dig', 'redhat.com./MX') }}
$ cat template.conf
...
Red Hat MX: 10 mx2.redhat.com.,5 mx1.redhat.com.

Conclusion

As shown the lookup plugin of Ansible provides many possibilities to integrate Ansible with existing tools and environments which already contain valuable data about the systems. It is easy to use, integrates well with the existing Ansible concepts and can quickly be integrated. Just drop it where a variable would be dropped, and it already works.

I am looking forward to more lookup modules support in the future – I’d love to see a generic “http” and a generic “SQL” plugin, even with the ability to provide credentials, although these features can be somewhat realized with already existing modules.

[Howto] Introduction to Ansible variables

Ansible Logo To become more flexible, Ansible offers the possibility to use variables in loops, but also to use information the target system provides.

Background

Ansible uses variables to enable more flexibility in playbooks and roles. They can be used to loop through a set of given values, access various information like the hostname of a system and replace certain strings in templates by system specific values. Variables are provided through the inventory, by variable files, overwritten on the command line and set in Tower.

But note that variable names have some restrictions in Ansible:

Variable names should be letters, numbers, and underscores. Variables should always start with a letter.

If the variable in question is a dictionary, a key/value pair, it can be referenced both by bracket and dot notation:

foo.bar
foo['bar']

The question is however: how to use variables? And how to get them in the first place?

Variables and loops

Loops are arguably among the most common use cases of variables in general – the same is true for Ansible. While they are not using variables provided externally, I’d like to start with loops to give a first idea how to use them. To copy a set of files it is either possible to write a task for each file or to just loop through them:

tasks:
  - name: copy files
    copy: src={{ item }} dest=/tmp/{{ item }}
    with_items:
      - alha
      - beta

This already shows simple basics: variables can be used in module arguments, and they are referenced via curly brackets {{ }}.

Variables and templates

More importantly, variables can be used to substitute keys in configuration files with system or run-time specific values. Imagine a file which containing the actual host name simply copied over from a central source to the clients. In case of one hundred machines there would be one hundred copies of the configuration file on the server, all almost identical except for the corresponding host name. That is a waste of time and space.

It’s better to have one copy of the configuration file, with a variable as place-maker instead of the host names – this is where templates come into play:

$ cat template.j2
My host name is {{ ansible_hostname }}.

The Ansible module to use templates and activate variable substitution is the template module:

tasks:
  - name: copy template
    template: src=template.j2 dest="/tmp/abcapp.conf"

When this task is run against a set of systems, the all get a file called abcapp.conf containing the individual host name of the given system.

This is again a simple example, but shows the basics: in templates variables are also referenced by {{ }}. Templates can become much more sophisticated, but I will try to cover that in another post.

Using variables in conditions

Variables can be used as conditions, thus ensuring that certain tasks are only run when for example on a given host the requested variable is set to a certain value:

tasks:
  - name: install Apache on Solaris
    pkg5: name=web/server/apache-24
    when: ansible_os_family == "Solaris"

  - name: install Apache on RHEL
    yum:  name=httpd
    when: ansible_os_family == "RedHat"

In this case, the first task is only applied on Red Hat Enterprise Linux servers, while the second is only run on Solaris machines.

Getting variables from the system

Using variables is one thing, but they need to be defined first. I have covered so far some use cases of variables, but not where to get them.

Ansible already defines a rich set of variables, individual for each system: whenever Ansible is run on a system all facts and information about the system are gathered and set as a variable. The available variables can be output via the setup module:

$ ansible neon -m setup
neon | success >> {
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
            "192.168.122.203"
        ], 
        "ansible_all_ipv6_addresses": [
            "fe80::5054:ff:feba:9db3"
        ], 
        "ansible_architecture": "x86_64", 
        "ansible_bios_date": "04/01/2014", 
...

All these variables can be used in templates, but also in the playbooks themselves as mentioned above in the conditions example. Roles can take advantage of these variables as well, of course.

Getting variables from the command line

Another way to define variables is to call Ansible playbooks with the option --extra-vars:

$ ansible-playbook --extra-vars "cli_var=production" system-setup.yml

The reference is again via the {{ }} brackets:

$ cat template.j2
environment: {{ cli_var }}.

Setting variables in playbooks

A more direct way is to define variables in playbooks directly, with the key vars:

...
hosts: all 
vars:
  play_var: bar 

tasks:
...

This can be extended by including an actual yaml file containing variables:

...
hosts: all
include_vars: setupvariables.yml

tasks:
...

Including further files with more variables comes in extremly handy when the variables for each system are kept in a system specific file, like $HOSTNAME.yml, because the included variable file can again be a variable:

...
hosts: all
include_vars: "{{ ansible_hostname }}.yml"

tasks:
...

In this case, the playbook reads in specific variables for each corresponding host – this way it is possible to set for example different virtual host names for Apache servers, or different backup systems in case of separated data centers.

Setting variables in the inventory

Sometimes it might make more sense to define specific variables in the already set up inventory:

[clients]
helium intevent_var=helium_123
neon invent_var=bar

The inventory more or less treats any argument which is not Ansible specific as a host variable.

It is also possible to set variables for entire host groups:

[clients]
helium
neon

[clients:vars]
invent_var=group-foo

Setting variables in the inventory makes sense when different teams use the same set of Ansible roles and playbooks but have different machine setups which need specific treatment at each location.

Setting variables on a system: local facts

Variables can also be set by dumping specialized files onto the system: when Ansible accesses a remote system it checks for the directory /etc/ansible/facts.d and all files ending in .fact are read. The files can be of various formats – INI, JSON, or even executables which return JSON code. This offers the possibility to add even generic fact providers via scripting.

For example:

$ cat /etc/ansible/facts.d/variables.fact 
[system]
foo=bar
dim=dum

The variables can be displayed via the setup module:

$ ansible neon -m setup
...
      "type": "loopback"
  }, 
  "ansible_local": {
      "variables": {
          "system": {
              "dim": "dum", 
              "foo": "bar"
          }
     }
  }, 
  "ansible_machine": "x86_64", 
...

Using the results of tasks: registered variables

The results of a task during a playbook run can also be stored inside a variable – together with conditionals this enables a playbook to react to the results of the given task with other tasks.

For example, given that an httpd service needs to run. If the service does not run, the entire server should be powered down immediately to ensure that no data corruption takes place. The corresponding playbook checks the httpd service, ignores errors but instead analyses the result and powers down the machine if the service cannot be started:

---
- name: register example
  hosts: all 
  sudo: yes

  tasks:
    - name: start service
      service: name=httpd state=started
      ignore_errors: True
      register: service_result

    - name: shutdown
      command: "shutdown -h +1m"
      when: service_result | failed

Another example would be to trigger a function to remove the host from the loadbalancer. Or to check if the database is available and would otherwise immediately close down the front firewall.

Accessing variables of other hosts

Last but not least it might be interesting to access the variables (read: facts) of other hosts. This can of course only be done if the facts are actually available.

If this is given, the data can be accessed via the hostvars key. The value for the key is the name given in the inventory.

Groups from within the inventory can also be accessed. That makes it possible to loop through a list of hosts in a group and for example gather all IP addresses or host names or other data.

The following template first accesses the host name of the machine “tower” and afterwards collects all the epoch data of all machines:

Tower name is: {{ hostvars['tower']['ansible_hostname'] }}
The epochs of the clients are:
{% for host in groups['clients'] %}
- {{ hostvars[host]['ansible_date_time']['epoch'] }}
{% endfor %}

The code for the loop is actual line statement of the Jinja2 template engine. The engine also offers if statements and other common operations.

This comes in handy in simple cases like filling the /etc/hosts, but can also empower admins to for example fill in the data of a loadbalancer or a firewall configuration.

Conclusion

Varialbes are a very powerful feature within Ansible to enrich the functionality it provides. Together with templates and the Jinja2 template and language engine the possibilities are almost endless. And sooner or later every admin will leave the simple Ansible calls and playbooks behind and start diving into variables, templates, and loops through variables of other hosts. To make system automation even easier.

[Howto] Adopting Ansible Galaxy roles for Solaris

Ansible LogoIt is pretty easy to manage Solaris with Ansible. However, the Ansible roles available at Ansible Galaxy usually target Linux based OS only. Luckily, adopting them is rather simple.

Background

As mentioned earlier Solaris machines can be managed via Ansible pretty well: it works out of the box, and many already existing modules are incredible helpful in managing Solaris installations.

At the same time, the Ansible Best Practices guide strongly recommends using roles to organize your IT with Ansible. Many roles are already available at the Ansible Galaxy ready to be used by the admin in need. Ansible Galaxy is a central repository for various roles written by the community.

However, Ansible Galaxy only recently added support for Solaris. There are currently hardly any roles with Solaris platform support available.

Luckily expanding existing Ansible roles towards Solaris is not that hard.

Example: Apache role

For example, the Apache role from geerlingguy is one of the highest rated roles on Ansible Galaxy. It installs Apache, starts the service, has support for vhosts and custom ports and is above all pretty well documented. Yet, there is no Solaris support right now… Although geerlingguy just accepted a pull request, so it won’t be long until the new version will surface at Ansible Galaxy.

The best way to adopt a given role for another OS is to extend the current role for an additional OS – in contrast to deleting the original OS support an replacing it by new, again OS specific configuration. This keeps the role re-usable on other OS and enables the community to maintain and improve a shared, common role.

With a bit of knowledge about how services are started and stopped on Linux as well as on Solaris, one major difference quickly comes up: on Linux usually the name of the controlled service is the exact same name as the one of the binary behind the service. The same name string is also part of the path to the usr files of the program and for example to the configuration files. On Solaris that is often not the case!

So the best is to check the given role if it starts or stops the service at any given point, if a variable is used there, and if this variable is used somewhere else but for example to create a path name or identify a binary.

The given example indeed controls a service. Thus we add another variable, the service name:

tasks/main.yml
@@ -41,6 +41,6 @@
 - name: Ensure Apache has selected state and enabled on boot.
   service:
-    name: "{{ apache_daemon }}"
+    name: "{{ apache_service }}"
     state: "{{ apache_state }}"
     enabled: yes

Next, we need to add the new variable to the existing OS support:

vars/Debian.yml
@@ -1,4 +1,5 @@
 ---
+apache_service: apache2
 apache_daemon: apache2
 apache_daemon_path: /usr/sbin/
 apache_server_root: /etc/apache2
vars/RedHat.yml
@@ -1,4 +1,5 @@
 ---
+apache_service: httpd
 apache_daemon: httpd
 apache_daemon_path: /usr/sbin/
 apache_server_root: /etc/httpd

Now would be a good time to test the role – it should work on the suported platforms.

The next step is to add the necessary variables for Solaris. The best way is to copy an already existing variable file and to modify it afterwards to fit Solaris:

vars/Solaris.yml
@@ -0,0 +1,19 @@
+---
+apache_service: apache24
+apache_daemon: httpd
+apache_daemon_path: /usr/apache2/2.4/bin/
+apache_server_root: /etc/apache2/2.4/
+apache_conf_path: /etc/apache2/2.4/conf.d
+
+apache_vhosts_version: "2.2"
+
+__apache_packages:
+  - web/server/apache-24
+  - web/server/apache-24/module/apache-ssl
+  - web/server/apache-24/module/apache-security
+
+apache_ports_configuration_items:
+  - regexp: "^Listen "
+    line: "Listen {{ apache_listen_port }}"
+  - regexp: "^#?NameVirtualHost "
+    line: "NameVirtualHost *:{{ apache_listen_port }}"

This specific role provides two playbooks to setup and configure each supported platform. The easiest way to create these two files for a new platform is again to copy existing ones and to modify them afterwards according to the specifics of Solaris.

The configuration looks like:

tasks/configure-Solaris.yml
@@ -0,0 +1,19 @@
+---
+- name: Configure Apache.
+  lineinfile:
+    dest: "{{ apache_server_root }}/conf/{{ apache_daemon }}.conf"
+    regexp: "{{ item.regexp }}"
+    line: "{{ item.line }}"
+    state: present
+  with_items: apache_ports_configuration_items
+  notify: restart apache
+
+- name: Add apache vhosts configuration.
+  template:
+    src: "vhosts-{{ apache_vhosts_version }}.conf.j2"
+    dest: "{{ apache_conf_path }}/{{ apache_vhosts_filename }}"
+    owner: root
+    group: root
+    mode: 0644
+  notify: restart apache
+  when: apache_create_vhosts

The setup thus can look like:

tasks/setup-Solaris.yml
@@ -0,0 +1,6 @@
+---
+- name: Ensure Apache is installed.
+  pkg5:
+    name: "{{ item }}"
+    state: installed
+  with_items: apache_packages

Last but not least, the platform support must be activated in the main/task.yml file:

tasks/main.yml
@@ -15,6 +15,9 @@
 - include: setup-Debian.yml
   when: ansible_os_family == 'Debian'
 
+- include: setup-Solaris.yml
+  when: ansible_os_family == 'Solaris'
+
 # Figure out what version of Apache is installed.
 - name: Get installed version of Apache.
   shell: "{{ apache_daemon_path }}{{ apache_daemon }} -v"

When you now run the role on a Solaris machine, it should install Apache right away.

Conclusion

Adopting a given role from Ansible Galaxy for Solaris is rather easy – if the given role is already prepared for multi OS support. In such cases adding another role is a trivial task.

If the role is not prepared for multi OS support, try to get in contact with the developers, often they appreciate feedback and multi OS support pull requests.

[Howto] Managing Solaris 11 via Ansible

Ansible LogoAnsible can be used to manage various kinds of Server operating systems – among them Solaris 11.

Managing Solaris 11 servers via Ansible from my Fedora machine is actually less exciting than previously thought. Since the amount of blog articles covering that is limited I thought it might be a nice challenge.

However, the opposite is the case: it just works. On a fresh Solaris installation, out of the box. There is not even need for additional configuration or additional software. Of course, ssh access must be available – but the same is true on Linux machines as well. It’s almost boring 😉

Here is an example to install and remove software on Solaris 11, using the new package system IPS which was introduced in Solaris 11:

$ ansible solaris -s -m pkg5 -a "name=web/server/apache-24"
$ ansible solaris -s -m pkg5 -a "state=absent name=/text/patchutils"

While Ansible uses a special module, pkg5, to manage Solaris packages, service managing is even easier because the usual service module is used for Linux as well as Solaris machines:

$ ansible solaris -s -m service -a "name=apache24 state=started"
$ ansible solaris -s -m service -a "name=apache24 state=stopped"

So far so good – of course things get really interesting if playbooks can perform tasks on Solaris and Linux machines at the same time. For example, imagine Apache needs to be deployed and started on Linux as well as on Solaris. Here conditions come in handy:

---
- name: install and start Apache
  hosts: clients
  vars_files:
    - "vars/{{ ansible_os_family }}.yml"
  sudo: yes

  tasks:
    - name: install Apache on Solaris
      pkg5: name=web/server/apache-24
      when: ansible_os_family == "Solaris"

    - name: install Apache on RHEL
      yum:  name=httpd
      when: ansible_os_family == "RedHat"

    - name: start Apache
      service: name={{ apache }} state=started

Since the service name is not the same on different operating systems (or even different Linux distributions) the service name is a variable defined in a family specific Yaml file.

It’s also interesting to note that the same Ansible module works different on the different operating systems: when a service is ordered to be stopped, but is not even available because the corresponding package and thus service definition is not even installed, the return code on Linux is OK, while on Solaris an error is returned:

TASK: [stop Apache on Solaris] ************************************************
failed: [argon] => {"failed": true}
msg: svcs: Pattern 'apache24' doesn't match any instances

FATAL: all hosts have already failed -- aborting

It would be nice to catch the error, however as far as I know error handling in Ansible can only specify when to fail, and not which messages/errors should be ignored.

But besides this problem managing Solaris via Ansible works smoothly for me. And it even works on Ansible Tower, of course:

Tower-Ansible-Solaris.png

I haven’t tried to install Ansible on Solaris itself, but since packages are available that shouldn’t be much of an issue.

So in case you have a mixed environment including Solaris and Linux machines (Red Hat, Fedora, Ubuntu, Debian, Suse, you name it) I can only recommend to start using Ansible as soon as you possible. It simply works and can ease the pain of day to day tasks substantially.

Ansible Galaxy just added Solaris platform support

Ansible LogoWhile Ansible is mostly used in Linux environments, it can also be used to manage other UNIX variants like Solaris. Now the central hub for Ansible roles, Ansible Galaxy, also added support for the platform Solaris.

Ansible is handy tool to manage multiple servers. Besides the usual Linux distributions it features support for BSD variants, Solaris and even Windows. However, the central hub to share Ansible roles, Ansible Galaxy, was still missing Solaris support until now: the support was added for version 10 as well as version 11.

That can already be seen when a role template is generated with the Galaxy tools:

$ ansible-galaxy init acme --force
- acme was created successfully
$ grep -B 1 -A 8 Solaris acme/meta/main.yml
  #  - any
  #- name: Solaris
  #  versions:
  #  - all
  #  - 10
  #  - 11.0
  #  - 11.1
  #  - 11.2
  #  - 11.3
  #- name: Fedora

This opens up the possibility to provide Ansible roles including Solaris support at a central place. Right now I already have a pull request to enable Solaris support on a very powerful Apache role. In a following blog report I’ll add the (surprisingly few) steps which were necessary to adjust the role to support Solaris.

It is great that Ansible Galaxy adds more and more platforms and thus broadens the usage of the central hub to cover more and more use cases. I’m looking forward to see more and more Solaris roles in the Galaxy. If you need help porting a role don’t hesitate to contact me.

The support is so far still in internal testing and will be made final when the above mentioned Github issue is closed.

[Howto] Accessing CloudForms updated REST API with Python

Red Hat CloudForms LogoCloudForms comes with a REST API which was updated to version 2.0 in CloudForms 3.2. It covers substantially more functions compared to v1.0 and also offers feature parity with the old SOAP interface. This post holds a short introduction to calling the REST API via Python.

Introduction

Red Hat CloudForms is a Manager to “manage virtual, private, and hybrid cloud infrastructures” – it provides a single interface to manage OpenStack, Amazon EC2, Red Hat Enterprise Virtualization Management, VMware vCenter and Microsoft System Center Virtual Machine Manager. Simply said it is a manager of managers. While CloudForms focusses on virtual environments of all kinds its abilities are not limited to simple deployment and starting/stopping VMs, but cover the entire business process and workflow surrounding larger deployments of virtual machines in large or distributed data centers. Tasks can be highly automated, charge backs and optimizer enable the admins to put workloads where they make most sense, an almost unbelievable amount of reports help operations and please the management and entire service catalogs can help provision not single VMs, but setups of interrelated instances. And of course, as with all Red Hat products and technologies, CloudForms is fully Open Source and based upon a community project: ManageIQ.

One of the major use cases of CloudForms is to keep an overview of all the various types of “clouds” used in production and the VMs running on them during the day to day work. Many companies have VMWare instances in their data centers, but also have a second virtual environment like RHEV or Hyper-V. Additionally they use public cloud offerings for example to cover the load during peak times, or integrate OpenStack to provide their own, private cloud. In such cases CloudForms is the one interface to rule them all, one interface to manage them. 😉

CloudForms itself can be managed via Webinterface but also via the API. Up until recently, the focus was on a SOAP API. Since Ruby on Rails – the base for CloudForms – will not support SOAP anymore in the future the developers decided to switch to a REST API. With CloudForms 3.2 this move was in so far completed that the REST API reached feature parity. The API offers quite a lot of functions – besides gathering information it can also be used to trigger actions, define or delete services, etc.

Python examples

In general the API can be called by any REST compatible tool – which means by almost any HTTP client. For my tests with the new API I decided to use Python and more specifically iPython together with the requests and the json library. All examples are surrounded by a JSON dumps statement to prettify the output.

The REST authentication is provided by default HTTP means. The normal way is to authenticate once, get a token in return and use the token for all further calls. The default API url is https://cf.example.com/api, which shows which collections can be queried via the API, for example: vms, clusters, providers, etc. Please note that the role based access control of CloudForms is also present in the API: you can only query collections and and modify objects when you have proper rights to do so.

current_token=json.loads(requests.get('https://cf.example.com/api/auth',auth=("admin",'password')).text)['auth_token']
print json.dumps(json.loads(requests.get('https://cf.example.com/api',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "collections": [
        {
            "description": "Automation Requests",
            "href": "https://cf.example.com/api/automation_requests",
            "name": "automation_requests"
        },
...

This shows that the basic access works. Next we want to query a certain collection, for example the vms:

print json.dumps(json.loads(requests.get('https://cf.example.com/api/vms',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
...
    "count": 2,
    "name": "vms",
    "resources": [
        {
            "href": "https://cf.example.com/api/vms/602000000000007"
        },
        {
            "href": "https://cf.example.com/api/vms/602000000000006"
        }
    ],
    "subcount": 2
}

While all VMs are listed, the information shown above are not enough to understand which vm is actually which: at least the name should be shown. So we need to expand the information about each vm and afterwards add a condition to only show name and for example vendor:: ?expand=resources&attributes=name,vendor:

print json.dumps(json.loads(requests.get('https://cf.example.com/api/vms?expand=resources&attributes=name,vendor',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
...
"resources": [
    {
        "href": "https://cf.example.com/api/vms/602000000000007",
        "id": 602000000000007,
        "name": "my-vm",
        "vendor": "redhat"
    },
    {
        "href": "https://cf.example.com/api/vms/602000000000006",
        "id": 602000000000006,
        "name": "myvm-clone",
        "vendor": "redhat"
    }
],

This works of course for 2 vms, but not if you manage 20.000. Thus it’s better to use a filter:&filter[]='name="my-vm"'. Since filters use a lot of quotation marks depending on the amount of strings you use it is best to define a string containing the filter argument and afterwards add that one to the URL:

filter="name='my-vm'"
print json.dumps(json.loads(requests.get('https://cf.example.com/api/vms?expand=resources&attributes=name&filter[]='+filter',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
...
"count": 2,
"name": "vms",
"resources": [
    {
        "href": "https://cf.example.com/api/vms/602000000000007",
        "id": 602000000000007,
        "name": "my-vm"
    }
],
"subcount": 1

Note the subcount which shows how many vms with the given name were found. If you want to combine more than one filter, simply add them to the URL: &filter[]='name="my-vm"'&filter[]='power_state=on'.

With a given href to the correct vm you can shut it down. Use the HTTP POST method and provide a JSON payload calling the action “stop”.

print json.dumps(json.loads(requests.post('https://cf.example.com/api/vms/602000000000007',headers={'X-Auth-Token' : current_token},,data=json.dumps({'action':'stop'})).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "href": "https://cf.example.com/api/vms/602000000000007",
    "message": "VM id:602000000000007 name:'my-vm' stopping",
    "success": true,
    "task_href": "https://cf.example.com/api/tasks/602000000000097",
    "task_id": 602000000000097
}

If you want to call an action to more than one instance, change the href to the corresponding collection and include the actual hrefs for the vms in the payload in an resources array:

print json.dumps(json.loads(requests.post('https://cf.example.com/api/vms',headers={'X-Auth-Token' : current_token},data=json.dumps({'action':'stop', 'resources': [{'href':'https://cf.example.com/api/vms/602000000000007'},{'href':'https://cf.example.com/api/vms/602000000000006'}]})).text),sort_keys=True,indent=4,separators=(',', ': '))
"results": [
    {
        "href": "https://cf.example.com/api/vms/602000000000007",
        "message": "VM id:602000000000007 name:'my-vm' stopping",
        "success": true,
        "task_href": "https://cf.example.com/api/tasks/602000000000104",
        "task_id": 602000000000104
    },
    {
        "href": "https://cf.example.com/api/vms/602000000000006",
        "message": "VM id:602000000000006 name:'myvm-clone' stopping",
        "success": true,
        "task_href": "https://cf.example.com/api/tasks/602000000000105",
        "task_id": 602000000000105
    }
]

The last example shows how more than one call to the API are connected to each other: we call the API to scan a VM, get the task id and query the task id to see if the task was successfully called. So first we call the API to start the scan:

print json.dumps(json.loads(requests.post('https://cf.example.com/api/vms/602000000000006',headers={'X-Auth-Token' : current_token},verify=False,data=json.dumps({'action':'scan'})).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "href": "https://cf.example.com/api/vms/602000000000006",
    "message": "VM id:602000000000006 name:'my-vm' scanning",
    "success": true,
    "task_href": "https://cf.example.com/api/tasks/602000000000106",
    "task_id": 602000000000106
}

Next, we take the given id 602000000000106 and query the state:

print json.dumps(json.loads(requests.get('https://cf.example.com/api/tasks/602000000000106',headers={'X-Auth-Token' : current_token},verify=False).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "created_on": "2015-08-25T15:00:16Z",
    "href": "https://cf.example.com/api/tasks/602000000000106",
    "id": 602000000000106,
    "message": "Task completed successfully",
    "name": "VM id:602000000000006 name:'my-vm' scanning",
    "state": "Finished",
    "status": "Ok",
    "updated_on": "2015-08-25T15:00:20Z",
    "userid": "admin"
}

However, please note that “Finished” here means that the call of the task was successful – but not necessarily the task outcome itself. For that you would have to call the vm state itself.

Final words

The REST API of CloudForms offers quite some useful functions to integrate CloudForms with your own programs, scripts and applications. The REST API documentation is also quite extensive, and the community documentation for ManageIQ has a lot of API usage examples.

So if you used to call your CloudForms via SOAP you will be happy to find the new REST API in CloudForms 3.2. If you never used the API you might want to start today – as you have seen its quite simple to get results quickly.