[Howto] Adopting Ansible Galaxy roles for Solaris

Ansible LogoIt is pretty easy to manage Solaris with Ansible. However, the Ansible roles available at Ansible Galaxy usually target Linux based OS only. Luckily, adopting them is rather simple.

Background

As mentioned earlier Solaris machines can be managed via Ansible pretty well: it works out of the box, and many already existing modules are incredible helpful in managing Solaris installations.

At the same time, the Ansible Best Practices guide strongly recommends using roles to organize your IT with Ansible. Many roles are already available at the Ansible Galaxy ready to be used by the admin in need. Ansible Galaxy is a central repository for various roles written by the community.

However, Ansible Galaxy only recently added support for Solaris. There are currently hardly any roles with Solaris platform support available.

Luckily expanding existing Ansible roles towards Solaris is not that hard.

Example: Apache role

For example, the Apache role from geerlingguy is one of the highest rated roles on Ansible Galaxy. It installs Apache, starts the service, has support for vhosts and custom ports and is above all pretty well documented. Yet, there is no Solaris support right now… Although geerlingguy just accepted a pull request, so it won’t be long until the new version will surface at Ansible Galaxy.

The best way to adopt a given role for another OS is to extend the current role for an additional OS – in contrast to deleting the original OS support an replacing it by new, again OS specific configuration. This keeps the role re-usable on other OS and enables the community to maintain and improve a shared, common role.

With a bit of knowledge about how services are started and stopped on Linux as well as on Solaris, one major difference quickly comes up: on Linux usually the name of the controlled service is the exact same name as the one of the binary behind the service. The same name string is also part of the path to the usr files of the program and for example to the configuration files. On Solaris that is often not the case!

So the best is to check the given role if it starts or stops the service at any given point, if a variable is used there, and if this variable is used somewhere else but for example to create a path name or identify a binary.

The given example indeed controls a service. Thus we add another variable, the service name:

tasks/main.yml
@@ -41,6 +41,6 @@
 - name: Ensure Apache has selected state and enabled on boot.
   service:
-    name: "{{ apache_daemon }}"
+    name: "{{ apache_service }}"
     state: "{{ apache_state }}"
     enabled: yes

Next, we need to add the new variable to the existing OS support:

vars/Debian.yml
@@ -1,4 +1,5 @@
 ---
+apache_service: apache2
 apache_daemon: apache2
 apache_daemon_path: /usr/sbin/
 apache_server_root: /etc/apache2
vars/RedHat.yml
@@ -1,4 +1,5 @@
 ---
+apache_service: httpd
 apache_daemon: httpd
 apache_daemon_path: /usr/sbin/
 apache_server_root: /etc/httpd

Now would be a good time to test the role – it should work on the suported platforms.

The next step is to add the necessary variables for Solaris. The best way is to copy an already existing variable file and to modify it afterwards to fit Solaris:

vars/Solaris.yml
@@ -0,0 +1,19 @@
+---
+apache_service: apache24
+apache_daemon: httpd
+apache_daemon_path: /usr/apache2/2.4/bin/
+apache_server_root: /etc/apache2/2.4/
+apache_conf_path: /etc/apache2/2.4/conf.d
+
+apache_vhosts_version: "2.2"
+
+__apache_packages:
+  - web/server/apache-24
+  - web/server/apache-24/module/apache-ssl
+  - web/server/apache-24/module/apache-security
+
+apache_ports_configuration_items:
+  - regexp: "^Listen "
+    line: "Listen {{ apache_listen_port }}"
+  - regexp: "^#?NameVirtualHost "
+    line: "NameVirtualHost *:{{ apache_listen_port }}"

This specific role provides two playbooks to setup and configure each supported platform. The easiest way to create these two files for a new platform is again to copy existing ones and to modify them afterwards according to the specifics of Solaris.

The configuration looks like:

tasks/configure-Solaris.yml
@@ -0,0 +1,19 @@
+---
+- name: Configure Apache.
+  lineinfile:
+    dest: "{{ apache_server_root }}/conf/{{ apache_daemon }}.conf"
+    regexp: "{{ item.regexp }}"
+    line: "{{ item.line }}"
+    state: present
+  with_items: apache_ports_configuration_items
+  notify: restart apache
+
+- name: Add apache vhosts configuration.
+  template:
+    src: "vhosts-{{ apache_vhosts_version }}.conf.j2"
+    dest: "{{ apache_conf_path }}/{{ apache_vhosts_filename }}"
+    owner: root
+    group: root
+    mode: 0644
+  notify: restart apache
+  when: apache_create_vhosts

The setup thus can look like:

tasks/setup-Solaris.yml
@@ -0,0 +1,6 @@
+---
+- name: Ensure Apache is installed.
+  pkg5:
+    name: "{{ item }}"
+    state: installed
+  with_items: apache_packages

Last but not least, the platform support must be activated in the main/task.yml file:

tasks/main.yml
@@ -15,6 +15,9 @@
 - include: setup-Debian.yml
   when: ansible_os_family == 'Debian'
 
+- include: setup-Solaris.yml
+  when: ansible_os_family == 'Solaris'
+
 # Figure out what version of Apache is installed.
 - name: Get installed version of Apache.
   shell: "{{ apache_daemon_path }}{{ apache_daemon }} -v"

When you now run the role on a Solaris machine, it should install Apache right away.

Conclusion

Adopting a given role from Ansible Galaxy for Solaris is rather easy – if the given role is already prepared for multi OS support. In such cases adding another role is a trivial task.

If the role is not prepared for multi OS support, try to get in contact with the developers, often they appreciate feedback and multi OS support pull requests.

Advertisements

[Howto] Managing Solaris 11 via Ansible

Ansible LogoAnsible can be used to manage various kinds of Server operating systems – among them Solaris 11.

Managing Solaris 11 servers via Ansible from my Fedora machine is actually less exciting than previously thought. Since the amount of blog articles covering that is limited I thought it might be a nice challenge.

However, the opposite is the case: it just works. On a fresh Solaris installation, out of the box. There is not even need for additional configuration or additional software. Of course, ssh access must be available – but the same is true on Linux machines as well. It’s almost boring 😉

Here is an example to install and remove software on Solaris 11, using the new package system IPS which was introduced in Solaris 11:

$ ansible solaris -s -m pkg5 -a "name=web/server/apache-24"
$ ansible solaris -s -m pkg5 -a "state=absent name=/text/patchutils"

While Ansible uses a special module, pkg5, to manage Solaris packages, service managing is even easier because the usual service module is used for Linux as well as Solaris machines:

$ ansible solaris -s -m service -a "name=apache24 state=started"
$ ansible solaris -s -m service -a "name=apache24 state=stopped"

So far so good – of course things get really interesting if playbooks can perform tasks on Solaris and Linux machines at the same time. For example, imagine Apache needs to be deployed and started on Linux as well as on Solaris. Here conditions come in handy:

---
- name: install and start Apache
  hosts: clients
  vars_files:
    - "vars/{{ ansible_os_family }}.yml"
  sudo: yes

  tasks:
    - name: install Apache on Solaris
      pkg5: name=web/server/apache-24
      when: ansible_os_family == "Solaris"

    - name: install Apache on RHEL
      yum:  name=httpd
      when: ansible_os_family == "RedHat"

    - name: start Apache
      service: name={{ apache }} state=started

Since the service name is not the same on different operating systems (or even different Linux distributions) the service name is a variable defined in a family specific Yaml file.

It’s also interesting to note that the same Ansible module works different on the different operating systems: when a service is ordered to be stopped, but is not even available because the corresponding package and thus service definition is not even installed, the return code on Linux is OK, while on Solaris an error is returned:

TASK: [stop Apache on Solaris] ************************************************
failed: [argon] => {"failed": true}
msg: svcs: Pattern 'apache24' doesn't match any instances

FATAL: all hosts have already failed -- aborting

It would be nice to catch the error, however as far as I know error handling in Ansible can only specify when to fail, and not which messages/errors should be ignored.

But besides this problem managing Solaris via Ansible works smoothly for me. And it even works on Ansible Tower, of course:

Tower-Ansible-Solaris.png

I haven’t tried to install Ansible on Solaris itself, but since packages are available that shouldn’t be much of an issue.

So in case you have a mixed environment including Solaris and Linux machines (Red Hat, Fedora, Ubuntu, Debian, Suse, you name it) I can only recommend to start using Ansible as soon as you possible. It simply works and can ease the pain of day to day tasks substantially.

Ansible Galaxy just added Solaris platform support

Ansible LogoWhile Ansible is mostly used in Linux environments, it can also be used to manage other UNIX variants like Solaris. Now the central hub for Ansible roles, Ansible Galaxy, also added support for the platform Solaris.

Ansible is handy tool to manage multiple servers. Besides the usual Linux distributions it features support for BSD variants, Solaris and even Windows. However, the central hub to share Ansible roles, Ansible Galaxy, was still missing Solaris support until now: the support was added for version 10 as well as version 11.

That can already be seen when a role template is generated with the Galaxy tools:

$ ansible-galaxy init acme --force
- acme was created successfully
$ grep -B 1 -A 8 Solaris acme/meta/main.yml
  #  - any
  #- name: Solaris
  #  versions:
  #  - all
  #  - 10
  #  - 11.0
  #  - 11.1
  #  - 11.2
  #  - 11.3
  #- name: Fedora

This opens up the possibility to provide Ansible roles including Solaris support at a central place. Right now I already have a pull request to enable Solaris support on a very powerful Apache role. In a following blog report I’ll add the (surprisingly few) steps which were necessary to adjust the role to support Solaris.

It is great that Ansible Galaxy adds more and more platforms and thus broadens the usage of the central hub to cover more and more use cases. I’m looking forward to see more and more Solaris roles in the Galaxy. If you need help porting a role don’t hesitate to contact me.

The support is so far still in internal testing and will be made final when the above mentioned Github issue is closed.

[Howto] Accessing CloudForms updated REST API with Python

Red Hat CloudForms LogoCloudForms comes with a REST API which was updated to version 2.0 in CloudForms 3.2. It covers substantially more functions compared to v1.0 and also offers feature parity with the old SOAP interface. This post holds a short introduction to calling the REST API via Python.

Introduction

Red Hat CloudForms is a Manager to “manage virtual, private, and hybrid cloud infrastructures” – it provides a single interface to manage OpenStack, Amazon EC2, Red Hat Enterprise Virtualization Management, VMware vCenter and Microsoft System Center Virtual Machine Manager. Simply said it is a manager of managers. While CloudForms focusses on virtual environments of all kinds its abilities are not limited to simple deployment and starting/stopping VMs, but cover the entire business process and workflow surrounding larger deployments of virtual machines in large or distributed data centers. Tasks can be highly automated, charge backs and optimizer enable the admins to put workloads where they make most sense, an almost unbelievable amount of reports help operations and please the management and entire service catalogs can help provision not single VMs, but setups of interrelated instances. And of course, as with all Red Hat products and technologies, CloudForms is fully Open Source and based upon a community project: ManageIQ.

One of the major use cases of CloudForms is to keep an overview of all the various types of “clouds” used in production and the VMs running on them during the day to day work. Many companies have VMWare instances in their data centers, but also have a second virtual environment like RHEV or Hyper-V. Additionally they use public cloud offerings for example to cover the load during peak times, or integrate OpenStack to provide their own, private cloud. In such cases CloudForms is the one interface to rule them all, one interface to manage them. 😉

CloudForms itself can be managed via Webinterface but also via the API. Up until recently, the focus was on a SOAP API. Since Ruby on Rails – the base for CloudForms – will not support SOAP anymore in the future the developers decided to switch to a REST API. With CloudForms 3.2 this move was in so far completed that the REST API reached feature parity. The API offers quite a lot of functions – besides gathering information it can also be used to trigger actions, define or delete services, etc.

Python examples

In general the API can be called by any REST compatible tool – which means by almost any HTTP client. For my tests with the new API I decided to use Python and more specifically iPython together with the requests and the json library. All examples are surrounded by a JSON dumps statement to prettify the output.

The REST authentication is provided by default HTTP means. The normal way is to authenticate once, get a token in return and use the token for all further calls. The default API url is https://cf.example.com/api, which shows which collections can be queried via the API, for example: vms, clusters, providers, etc. Please note that the role based access control of CloudForms is also present in the API: you can only query collections and and modify objects when you have proper rights to do so.

current_token=json.loads(requests.get('https://cf.example.com/api/auth',auth=("admin",'password')).text)['auth_token']
print json.dumps(json.loads(requests.get('https://cf.example.com/api',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "collections": [
        {
            "description": "Automation Requests",
            "href": "https://cf.example.com/api/automation_requests",
            "name": "automation_requests"
        },
...

This shows that the basic access works. Next we want to query a certain collection, for example the vms:

print json.dumps(json.loads(requests.get('https://cf.example.com/api/vms',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
...
    "count": 2,
    "name": "vms",
    "resources": [
        {
            "href": "https://cf.example.com/api/vms/602000000000007"
        },
        {
            "href": "https://cf.example.com/api/vms/602000000000006"
        }
    ],
    "subcount": 2
}

While all VMs are listed, the information shown above are not enough to understand which vm is actually which: at least the name should be shown. So we need to expand the information about each vm and afterwards add a condition to only show name and for example vendor:: ?expand=resources&attributes=name,vendor:

print json.dumps(json.loads(requests.get('https://cf.example.com/api/vms?expand=resources&attributes=name,vendor',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
...
"resources": [
    {
        "href": "https://cf.example.com/api/vms/602000000000007",
        "id": 602000000000007,
        "name": "my-vm",
        "vendor": "redhat"
    },
    {
        "href": "https://cf.example.com/api/vms/602000000000006",
        "id": 602000000000006,
        "name": "myvm-clone",
        "vendor": "redhat"
    }
],

This works of course for 2 vms, but not if you manage 20.000. Thus it’s better to use a filter:&filter[]='name="my-vm"'. Since filters use a lot of quotation marks depending on the amount of strings you use it is best to define a string containing the filter argument and afterwards add that one to the URL:

filter="name='my-vm'"
print json.dumps(json.loads(requests.get('https://cf.example.com/api/vms?expand=resources&attributes=name&filter[]='+filter',headers={'X-Auth-Token' : current_token}).text),sort_keys=True,indent=4,separators=(',', ': '))
...
"count": 2,
"name": "vms",
"resources": [
    {
        "href": "https://cf.example.com/api/vms/602000000000007",
        "id": 602000000000007,
        "name": "my-vm"
    }
],
"subcount": 1

Note the subcount which shows how many vms with the given name were found. If you want to combine more than one filter, simply add them to the URL: &filter[]='name="my-vm"'&filter[]='power_state=on'.

With a given href to the correct vm you can shut it down. Use the HTTP POST method and provide a JSON payload calling the action “stop”.

print json.dumps(json.loads(requests.post('https://cf.example.com/api/vms/602000000000007',headers={'X-Auth-Token' : current_token},,data=json.dumps({'action':'stop'})).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "href": "https://cf.example.com/api/vms/602000000000007",
    "message": "VM id:602000000000007 name:'my-vm' stopping",
    "success": true,
    "task_href": "https://cf.example.com/api/tasks/602000000000097",
    "task_id": 602000000000097
}

If you want to call an action to more than one instance, change the href to the corresponding collection and include the actual hrefs for the vms in the payload in an resources array:

print json.dumps(json.loads(requests.post('https://cf.example.com/api/vms',headers={'X-Auth-Token' : current_token},data=json.dumps({'action':'stop', 'resources': [{'href':'https://cf.example.com/api/vms/602000000000007'},{'href':'https://cf.example.com/api/vms/602000000000006'}]})).text),sort_keys=True,indent=4,separators=(',', ': '))
"results": [
    {
        "href": "https://cf.example.com/api/vms/602000000000007",
        "message": "VM id:602000000000007 name:'my-vm' stopping",
        "success": true,
        "task_href": "https://cf.example.com/api/tasks/602000000000104",
        "task_id": 602000000000104
    },
    {
        "href": "https://cf.example.com/api/vms/602000000000006",
        "message": "VM id:602000000000006 name:'myvm-clone' stopping",
        "success": true,
        "task_href": "https://cf.example.com/api/tasks/602000000000105",
        "task_id": 602000000000105
    }
]

The last example shows how more than one call to the API are connected to each other: we call the API to scan a VM, get the task id and query the task id to see if the task was successfully called. So first we call the API to start the scan:

print json.dumps(json.loads(requests.post('https://cf.example.com/api/vms/602000000000006',headers={'X-Auth-Token' : current_token},verify=False,data=json.dumps({'action':'scan'})).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "href": "https://cf.example.com/api/vms/602000000000006",
    "message": "VM id:602000000000006 name:'my-vm' scanning",
    "success": true,
    "task_href": "https://cf.example.com/api/tasks/602000000000106",
    "task_id": 602000000000106
}

Next, we take the given id 602000000000106 and query the state:

print json.dumps(json.loads(requests.get('https://cf.example.com/api/tasks/602000000000106',headers={'X-Auth-Token' : current_token},verify=False).text),sort_keys=True,indent=4,separators=(',', ': '))
{
    "created_on": "2015-08-25T15:00:16Z",
    "href": "https://cf.example.com/api/tasks/602000000000106",
    "id": 602000000000106,
    "message": "Task completed successfully",
    "name": "VM id:602000000000006 name:'my-vm' scanning",
    "state": "Finished",
    "status": "Ok",
    "updated_on": "2015-08-25T15:00:20Z",
    "userid": "admin"
}

However, please note that “Finished” here means that the call of the task was successful – but not necessarily the task outcome itself. For that you would have to call the vm state itself.

Final words

The REST API of CloudForms offers quite some useful functions to integrate CloudForms with your own programs, scripts and applications. The REST API documentation is also quite extensive, and the community documentation for ManageIQ has a lot of API usage examples.

So if you used to call your CloudForms via SOAP you will be happy to find the new REST API in CloudForms 3.2. If you never used the API you might want to start today – as you have seen its quite simple to get results quickly.