[Short Tip] Add a path entry to Nushell

Sometimes, if you cannot properly transfer files to your phone, try another mtp implementation!

Adding a path in nushell is pretty straight forward: the configuration is done in ~/config/nu/config.toml in the [path] section.

If you don’t have it, make sure that the default entries are listed there as well when you start bringing in your own directories. The fastest way to populate your configuration with the default entries is to ask nushell to do it: config set path $nu.path

Next, add the directories you need:

path = ["/usr/local/bin", "/usr/local/sbin", "/usr/bin", "/usr/sbin","/home/rwolters/go/bin"]

In the above example I added the default go binary directory to the list.

[Short Tip] Define an alias in Nushell

A few days ago I decided to switch my main shell to nushell. It offers next generation capabilities similar to other modern shells like Powershell:

Rather than thinking of files and services as raw streams of text, Nu looks at each input as something with structure. For example, when you list the contents of a directory, what you get back is a table of rows, where each row represents an item in that directory.

Nushell Philosophy

This switch feels like the biggest shell related change for me since I switched from Windows Command Prompt to Bash. Even the switch from Bash to ZSH was small in comparison.

As part of this I have to relearn how to do a lot of things. For example, defining an alias is totally different.

The nushell configuration takes place in ~.config/nu/config.toml. To add an alias there, add the section startup if it is not there already, and add a list item there:

startup = [
    "alias evince = flatpak run org.gnome.Evince",
    "alias ll = ls -la",
    ]

In the above example two aliases are given, one to call ls, and one for launching evince via flatpaks.

Image by julia roman from Pixabay

Useful command line options for ansible-playbook

Ansible LogoAnsible provides quite some useful command line options. Most of them are especially interesting during debugging.

Background

There are three major ways to work with Ansible:

  • launching single tasks with the ansible command
  • executing playbooks viaansible-playbook
  • using Tower to manage and run playbooks

While Tower might be the better option to run Ansible in the day-to-day business, and the ansible CLI itself is most likely only in one-time runs used, the executing of playbooks on the command line often happens during the development of playbooks, when no Tower is available – or during debugging. In such cases, there are quite some useful command line options which might not even be known to the seasoned Ansible user.

Do I say this right? – Syntax checking

Playbooks are written in YAML, and in YAML syntax is crucial – especially indentation:

Data structure hierarchy is maintained by outline indentation.

To check if a playbook is correctly formatted, the option --syntax-check looks at all involved playbooks and verifies the correct syntax. During a syntax check, no playbooks are actually executed.

$ ansible-playbook --syntax-check oraclejdk-destroy.yml
ERROR! Syntax Error while loading YAML.

The error appears to have been in '/home/liquidat/Gits/github/ansible-demo-oraclejdk/oracle-windows-destroy.yml': line 10, column 11,
but may be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  win_template: src=data/remove-program.j2 dest=C:\\temp\\remove-program.ps1
    - name: remove application
          ^ here

The syntax check helps if a playbook fails for no apparent reason – or if a playbook was edited a lot and it is simply not sure if everything was moved around correctly.

Whom am I talking to? – Listing affected hosts

With complex playbooks and dynamic inventories it sometimes is hard to say against which hosts a playbook will actually be executed. In such cases, the option --list-hosts will output a list of affected hosts, including the name of the actual play and the pattern with which the hosts were chosen:

$ ansible-playbook --list-hosts oraclejdk-destroy.yml

playbook: oraclejdk-destroy.yml

  play #1 (windows): remove OracleJDK on Windows	TAGS: []
    pattern: [u'windows']
    hosts (1):
      radon

  play #2 (rhel): remove OracleJDK on RHEL	TAGS: []
    pattern: [u'rhel']
    hosts (2):
      neon
      helium
...

This works also together with the -l option and might help debugging your inventory.

Again, no tasks are actually execute when the list of hosts is queried.

What’s going on here? – List tasks

Another thing which can get pretty complicated is the list of tasks actually executed: think of complex playbooks including other complex playbooks. That can get pretty complex and difficult to understand – here the option --list-tasks comes in handy. It lists what will be done, showing the names of the tasks but not executing any of them on the target nodes:

$ ansible-playbook --list-tasks oraclejdk-destroy.yml

playbook: oraclejdk-destroy.yml

  play #1 (windows): remove OracleJDK on Windows	TAGS: []
    tasks:
      copy Java remove script to temp	TAGS: []
      remove application	TAGS: []
      remove temp dir in Windows	TAGS: []

  play #2 (rhel): remove OracleJDK on RHEL	TAGS: []
    tasks:
      remove java dir	TAGS: []
...

What’s that thing? – List all tags

Besides all tasks, the used tags can be listed as well.

$ ansible-playbook --list-tags setup-control.yml

playbook: setup-control.yml

  play #1 (tuzak): 	TAGS: []
      TASK TAGS: [base_setup, db, imap, ldap, mail, oc, smtp]
...

Again, this option helps providing an overview what a playbook has to offer, how to use it. And again this option does not execute any task on the target node.

Are you sure? – Running in test mode

Ansible provides a so called check mode, also called dry run mode (in Tower for example). Invoked via --check the check mode does not alter the target nodes, but tries to output what would change and what not. Note however that this needs to be supported by the used modules, and not all modules support this.

For example, the following listing shows several tasks not supporting the dry run, which is indicated by the “skipping” line.

$ ansible-playbook --check oraclejdk-setup.yml

PLAY [set up OracleJDK on Windows] *********************************************

TASK [setup] *******************************************************************
ok: [radon]

TASK [set up temp dir in Windows] **********************************************
skipping: [radon]

TASK [copy JDK to Windows client] **********************************************
skipping: [radon]

TASK [run exe installer] *******************************************************
skipping: [radon]
...
PLAY [set up OracleJDK on RHEL] ************************************************

TASK [setup] *******************************************************************
ok: [helium]
ok: [neon]

TASK [copy JDK to RHEL client] *************************************************
skipping: [helium]
skipping: [neon]
....

This is quite useful to get an idea what impact the run of a playbook might have on target nodes. The lack of support in several modules dampens the positive effect a bit, though.

But since the --diff option (see below) supports it, it can be quite handy in certain situations.

Let me have a look at that… – Going through tasks step by step

Imagine that a playbook runs without errors, but somehow the result is not what exactly what was expected. In such cases one way to debug everything is to go through each task at a time, step by step, checking the state of all involved components after each task. This can be done with the option --step.

$ ansible-playbook --step oraclejdk-setup.yml

PLAY [set up OracleJDK on Windows] *********************************************
Perform task: TASK: setup (y/n/c): y

Perform task: TASK: setup (y/n/c): *********************************************

TASK [setup] *******************************************************************
ok: [radon]
Perform task: TASK: set up temp dir in Windows (y/n/c): y

Perform task: TASK: set up temp dir in Windows (y/n/c): ************************

TASK [set up temp dir in Windows] **********************************************
changed: [radon]
Perform task: TASK: copy JDK to Windows client (y/n/c): 
...

This is incredibly helpful on complex setups involving multiple nodes.

And yes, this time the tasks are actually executed on the target node!

Get me right there! – Starting playbooks in the middle

During debugging and development it might make sense to start playbooks not at the beginning, but somewhere in between. For example, because a playbook failed at task 14, and you don’t want to go through the first 13 tasks again. Starting at a given task requires the appropriate name of the task – and the option --start-at-task:

$ ansible-playbook --start-at-task="run exe installer" oraclejdk-setup.yml

PLAY [set up OracleJDK on Windows] *********************************************

TASK [setup] *******************************************************************
ok: [radon]

TASK [run exe installer] *******************************************************
ok: [radon]
...

In this example, the two tasks “set up temp dir in Windows” and “copy JDK to Windows client” are skipped, and the playbook starts directly at “run exe installer”. Note that skipped tasks are not shown or listed at all, and that the setup is run nevertheless.

As shown above, the proper name of each task is listed with the --list-tasks option.

Get down to business! – Showing diffs

Ansible is often used to deploy files, especially using templates. Usually, when a file is changed, Ansible just highlights that a change occurred – but not what was actually changed. In such cases, the option --diff comes in handy: it shows the diff in typical patch form:

$ ansible-playbook --diff examples/template.yml

PLAY [template example] ********************************************************

TASK [setup] *******************************************************************
ok: [helium]

TASK [copy template] ***********************************************************
changed: [helium]
--- before: /tmp/template.conf
+++ after: dynamically generated
@@ -1,2 +1,3 @@
 hostname: ansible-demo-helium
-bumble: bee
+foo: bar
+MX: 10 mx2.redhat.com.,5 mx1.redhat.com.

PLAY RECAP *********************************************************************
helium                     : ok=2    changed=1    unreachable=0    failed=0

This can be even combined with the option --check: in such cases, the diff is printed, but the change is not performed on the target node. That’s pretty handy indeed

That was interesting! – Summary

To summarize, ansible-playbook has quite some options to help debugging playbooks. The fact that many do not alter the target nodes makes it possible to use them on productive systems as well (but with care, as always). They also help a lot when it comes to understanding unknown playbooks, for example from other departments or coworkers.

[Short Tip] verify YAML in Shell via Python one-liner [Update]

python logo

Today the question came up how to verify YAML files easily. Of course, there are many very good online parser. But I was wondering if it is possible to do it simply in Bash/ZSH, using a Python one-liner. Here is the code:

$ python -c 'import yaml,sys;yaml.safe_load(sys.stdin)' < yamltest.txt

It throws an exception if the file is not a proper (aka importable) YAML file. Otherwise it just returns with a 0 exit code.

Please note that I am not sure how tolerant yaml.safe_load is. And note that PyYAML needs to be installed.

Update:
Updated to avoid cat abuse – safe the kittens! Thanks to ichor!

[Short Tip] show processes accessing a file: fuser & lsof

920839987_135ba34fff

Sometimes it’s good to know which processes access certain files, paths or devices. Think of debugging, but also of certain files blocked due to currently running processes. There are two commonly used tools to get information about which process accesses which file: lsof and fuser.

The best known tool is arguably lsof – it lists all open files, the output can be limited with options and arguments. Typical commands are:

$ lsof -p $PROCESSID
$ lsof $FILENAME
$ lsof |grep $EXPRESSION

The other well known tool is fuser. Compared to lsof it works the other way around: it lists open processes for a given target. The target must always be supplied and is not optional. Typical commands are:

$ fuser $FILENAME
$ fuser -mva $MOUNTPOINT
$ fuser -k $FILENAME

The last command is used to kill the process directly.

In the day to day work fuser feels slightly less bloated and is more handy to simply kill processes right away. On the other hand, if you are used to lsof there is hardly any reason to switch to fuser.