When working with web services, especially REST APIs, Ansible can be of surprising help when you need to automate those, our want to integrate them into your automation.
However, today I run into a strange Python bug while I tried to use the uri module:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: can't concat str to bytes
fatal: [localhost]: FAILED! => {"changed": false, "content": "", "elapsed": 0, "msg": "Status code was -1 and not [200]: An unknown error occurred: can't concat str to bytes", "redirected": false, "status": -1, "url": "https://www.ansible.com"}
This drove me almost nuts because it happened on all kinds of machines I tested, even with Ansible’s devel upstream version. It was even independent of the service I targeted – the error happened way earlier. And a playbook to showcase this was suspiciously short and simple:
---
- name: Show concat str byte error
hosts: localhost
connection: local
gather_facts: no
tasks:
- name: call problematic URL call
uri:
url: "https://www.ansible.com"
method: POST
body_format: json
body:
name: "myngfw"
When I was about to fill an issue at Ansible’s Github page I thought again and wondered that this is too simple: I couldn’t imagine that I was the only one hitting this problem. I realized that the error had to be on my side. And thus meant that something was missing.
And indeed: the body_format option was not explicitly stated, so Ansible assumed “raw”, while my body data were provided in json format. A simple
Did you ever saw a git diff with file changes you wanted to omit? I recently had to go through a very large git diff – and I realized that some modified files were not needed and I had to somehow remove them from the diff.
To go through the huge diff and remove all patches to certain files manually would be way too much work. Luckily since Git 1.9 you can use pathspec patterns to limit the output of git – and thus the output of git-diff:
The exclamation mark is just a short form for (exclude). If you use the bracket style you can even add additional “magic words” – for example to make the exclusion case insensitive:
Sometimes it happens that SSH connections lock up. For example due to weird SSH server configuration or bad connectivity on your side, suddenly your SSH connection is broken. You cannot send any more comments via the SSH connection. The terminal just doesn’t react.
And that includes the typical exit commands: Ctrl+z or Ctrl+d are not working anymore. So you are only left with the choice to close the terminal – right? In fact, no, you can just exist the SSH session.
The trick is: Enter+~+.
Why does this work? Because it is one of the defined escape sequences:
The supported escapes (assuming the default ‘~’) are: ~.Disconnect. ~^Z Background ssh. ~# List forwarded connections. ~& Background ssh at logout when waiting for forwarded connection / X11 sessions to terminate. […]
The tool tower-cli is often used to pre-configure Ansible Tower in a scripted way. It provides a convenient way to boot-strap a Tower configuration. But adding SSH keys as machine credentials is far from easy.
Boot-strapping Ansible Tower can become necessary for testing and QA environments where the same setup is created and destroyed multiple times. Other use cases are when multiple Tower installations need to be configured in the same way or share at least a larger part of the configuration.
One of the necessary tasks in such setups is to create machine credentials in Ansible Tower so that Ansible is able to connect properly to a target machine. In a Linux environment, this is often done via SSH keys.
However, tower-cli calls the Tower API in the background – and JSON POST data need to be in one line. But SSH keys come in multiple lines, so providing the file via a $(cat ssh_file) does not work:
But for my use case I needed to read an existing SSH file directly, and did not want to add another manual step or create an additional variables file. The trick is a rather complex piece of SED:
This method is appealing since the corresponding sed call is a little bit easier to understand. But make sure to indent the variables exactly like shown above.
I prefer self hosted solution for some tasks. But this also means that I have to troubleshoot my problems on my own. Recently a go-ldap error gave me a headache. Here is the analysis of the protocol error – and how to solve it.
For certain projects I prefer a self hosted Git server. Solutions like Gitea, the fast developing and striving fork of Gogs, make this painless and easy to do – especially in a containerized environment.
My users are managed in a FreeIPA, and Gitea connects to it via LDAP. And this is a constant source for trouble. Gitea is written in Go, and the go ldap libraries seem to be far from perfect.
For example, after a recent update of my environment, login at Gitea stopped working:
Since LDAP login still worked fine with other tools I assumed a problem in the new Gitea version and filled a bug report. Other users with the same problem joined soon after, but no one was able to provide a solution.
After some research I figured out that the problem appeared to be related to an update in the FreeIPA server: a security update in the underlying 389 server lead to protocol errors when empty attributes were part of the request.
An updated version of the go-ldap library was supposed to fix this – and indeed, after Gitea updated the library other users reported that the issue was fixed for them.
However, not for me: I still had the problem, and got frustrated over this for weeks.
It took me another evening of research until I found the important missing detail: a Grafana user had the same problem. The updated library did not help there either. But reducing the number of empty attributes by providing values for default, thus otherwise empty attributes did the trick:
For me the error occurs when I have less than 4 attributes.
In the end they figured out that one empty attribute was ok to be sent, but not two. With this information, the fix of my problem was easy: I previously had not set the attribute for First Name and Surname. I added those and immediately was able to login again.
Gitea with LDAP attributes
So: if you ever run into the same problem with Go and LDAP, check if you are indeed sending more than one empty attribute!
This was the second LDAP problem I encountered using Gitea. Due to this experience I do not really feel comfortable with using this combination – and will not be surprised if it breaks again with the next update.
On the other hand LDAP is a rather complicated protocol and probably a bit overkill for a simple use case like this. If I ever re-do my setup I might change over to mail based authentication.
Ansible uses the Jinja2 template engine to handle variables. This includes the default filter, which sets a default value if a referenced variable is not explicitly defined somewhere else.
With Ansible it might happen that instead of a skalar variable a key-value is needed, a dictionary. If you just paste the plain text in there, you might run into trouble:
fatal: [test.example.com]: FAILED! => {"changed": false, "msg": "argument env is of type and we were unable to convert to dict: dictionary requested, could not parse JSON or key=value"}
The key-value pair needs to be properly formatted:
Ansible Galaxy recently got a fresh update and now has much more features worth a look. Among those are automatic quality scorings.
In a recent role upload my scoring was only 4.5. One of the problems was a “invalid platform”. I wondered which platforms are supported, and how the strings for those are, but the documentation is sparse in this regard.
However, Ansible Galaxy does feature an API to query those things. And in fact galaxy.ansible.com/api/v1/platforms/ shows the appropriate Fedora versions: