Improving Ansible

The reason for this article was a post in @pro_ansible Telegram chat:

Vladislav? Shishkov, [17.02.21 20:59] Gentlemen, there are two questions regarding a custom long operation, for example, a backup: 1. Is it possible to tighten the progress bar of a custom bash through the ansible? (if through a plugin, then kick into some example or documentation pliz) 2. It seems like you want to write a plugin for this bash, but the question arises, how to be and how to solve the moments of execution that are idempotent?

A quick search in the backyard of memory did not suggest anything suitable. Nevertheless, I definitely remembered that the Ansible code is easy to read, and it supports extension by both plugins and regular Python modules out-of-the-box. Given so, nothing could prevents to once again push the boundaries of the possible. Hold my beer!

It is perfectly clear that standard Ansible already knows how to do both steps, only the resulting stdout «exhaust» is collected into a single whole and transmitted to the control host after the end of the process, but now we want to do this in real time instead of somewhen. Therefore, one can at least look at the existing implementation, and as a maximum - somehow reuse the existing code.

Indeed, the original question can be boiled down to two simple steps:

1. Capture stdout of a command at the target host;

2. Send the captured data to the management host.

Transferring the data to the control host

I suggest starting from the very end: we need to establish an additional transmission channel to the control host. The solution to this question looks quite obvious: it's just enough to remember that Ansible runs on top of ssh, and use the port forwarding function:

# let's put it somewhere near:
# https://github.com/ansible/ansible/blob/5078a0baa26e0eb715e86c93ec32af6bc4022e45/lib/ansible/plugins/connection/ssh.py#L662
self._add_args(
    b_command,
    (b"-R", b"127.0.0.1:33333:" + to_bytes(self._play_context.remote_addr, errors='surrogate_or_strict', nonstring='simplerepr') + b":33335"),
    u"ANSIBLE_STREAMING/streaming set"
)

How does it work? When assembling the command line arguments to establish an ssh connection into a single bunch, this piece of code will provide us on the target host with port 33333 at 127.0.0.1, which will tunnel incoming connections directly to the port 33335 of the controller.

For the sake of easyness, let's use netcat (indeed, what is an article without a cat?): nc -lk 33335

At this point, by the way, you can already start Ansible and check that the tunnel works as it should: although nothing is being transmitted through it yet, we can already go to the console on the target host and execute nc 127.0.0.1 33333, then enter some phrase and see it as an output of the command listening the 33335 port above.

Stdout interception

Half the battle is done - let's move on. We want to intercept the stdout of some command - according to the logic of Ansible's work, the "shell" module is suitable for us. It's funny that it turned out to be a dummy - there is no line of code in it, except for documentation and examples, but we find a reference to the "command" module within. Everything turned out well with it, except for the fact that the required function is not directly described here, although it is called. Anyway, it's almost the bull's-eye hit, because in the end its declaration was found in another file.

Recall the instant juice ads slogan “Just add some water” and add a pinch of code:

# In the very beginning of basic.py, together with other imports 
import socket

# in run_command - somewhere near:
# https://github.com/ansible/ansible/blob/5078a0baa26e0eb715e86c93ec32af6bc4022e45/lib/ansible/module_utils/basic.py#L2447
clientSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM);
clientSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
clientSocket.connect(("127.0.0.1",33333));

# in run_command - somewhere near:
# https://github.com/ansible/ansible/blob/5078a0baa26e0eb715e86c93ec32af6bc4022e45/lib/ansible/module_utils/basic.py#L2455
clientSocket.send(b_chunk);

# in run_command - somewhere near:
# https://github.com/ansible/ansible/blob/5078a0baa26e0eb715e86c93ec32af6bc4022e45/lib/ansible/module_utils/basic.py#L2481
clientSocket.close()

Putting it together and launching

What is left to do? That's right, work out the way to connect the modified modules to the stock Ansible. Just in case, I remind you: we have changed one connection plugin, and one module from the Ansible standard library. I guess experienced fighters don't even seem to need these explanations, but the very beginners in these dark pathways are sort of helpless. OK then, just for you - create and "module_utils" folder on the same directory level as your playbook, and copy modified basic.py there. Then add "connection_plugins" dir on the same dir level, and copy modified ssh.py there.

So, this is the time. The result in the form of a static image is not very revealing, so I set up tmux and started recording a screencast.