Framework for our automation bots.

Originally we started working on bots without any central infrastructure. We reached a point where we realized that the software does not scale and it's hard to maintain. Therefore this project should be focused on following capabilities:

• Security hardening
  • don't run as root in the container
  • don't store passwords in the containers
  • dedicated identity for bots
• Use ansible/makefiles for automation
• Automate build & release & deploy process
• 100% automated
  • ideally via dist-git & osbs (eat own dog food)
• images are stored in a registry
• Validation is in-place
• staging environment
• Monitoring
  • seamless monitoring - when something goes wrong, we are first to know that and we know relevant info to resolve the issue ASAP
  • metrics - have a dashboard where we can see statistics
• Container entrypoint scripts
  • ideally have no entrypoints scripts and keep all the logic in bots

Proof of concept of using Celery (asynchronous task queue based on distributed message passing) to organize bots.

There's a docker-compose.yml to test it locally or scripts to deploy to OpenShift.

It's configured to use Redis as broker and backend

We use docker-compose, so you have to install it first. To try Frambo locally, start our example bot with make example-bot-start. Run make example-bot-build before first run, example-bot-start doesn't depend on it to not rebuild with each tiny change. It starts one celery worker, listening on queue defined in docker-compose.yml. In other terminal verify that it works correctly by running make example-bot-run-task, which sends one task to the queue.

There's also Flower (monitoring tool), which runs on http://localhost:5555. After running make example-bot-run-task you should see one succeeded task there.

To check content of the redis instance there's redis-commander running on http://localhost:8081.

Some parts of frambo, for example some urls, hostnames or bot names are configurable in frambo/data/conf.d/config.yml file. The config file should contain a dictionary where each item key is a module name and value can be either a dictionary or a list of dicts. If it's a list, then each item has to contain a 'deployment' and config parser will select item whose 'deployment' matches DEPLOYMENT environment variable value.

See examples/bot/ directory for example bot implementation. If you have any questions don't hesitate to ask and we'll update the documentation with answers.

Frambo provides couple variables from pagure.py:

• PAGURE_HOST ... used for clonning repositories and execution git operations
• PAGURE_URL ... refers to https://{PAGURE_HOST}/

In your bot, you can e.g. ask for username:

import requests

from frambo.fedora_pagure import PAGURE_URL

url = f"{PAGURE_URL}/api/0/-/whoami"
r = requests.post(url, headers={'Authorization': 'token {}'.format(pagure_api_token)})

where pagure_api_token is taken from token page

You can easily verify locally that bot-cfg.yaml for your repository is valid.

$ git clone https://github.com/user-cont/frambo
$ cd frambo
$ make validate-bot-cfg BOT_CFG_PATH=<PATH>