- Overview
- Module Description - What the module does and why it is useful
- Setup - The basics of getting started with airflow
- Usage - Configuration options and additional functionality
- Reference - An under-the-hood peek at what the module is doing and how
- Development - Guide for contributing to the module
This module manages airflow by Airbnb.
The airflow module sets up and configures airflow.
This module has been tested against airflow versions: 1.5.2, 1.6.2
This module does not initialize the airflow database schema - you can do so by executing:
airflow initdb
More info here.
The module has been tested on CentOS 7
- Airflow package.
- Airflow configuration file.
- Airflow services.
- Airflow templates.
- This module does not install: MySQL, PostgreSQL, RabbitMQ or Redis - You have to use other modules for them. We use puppetlabs/mysql, puppetlabs/rabbitmq and garethr/erlang.
Please refer to airflow installation before using this module.
airflow module depends on the following puppet modules:
- puppetlabs-stdlib >= 1.0.0
- stankevich-python >= 1.9.8
- camptocamp-systemd >= 0.2.2
Install this module via any of these approaches:
- librarian-puppet
- git-submodule
puppet module install similarweb-airflow
class { 'airflow':
version => '1.6.2',
home_folder => '/usr/local/airflow'
}
class { 'airflow': } ->
class { 'airflow::service::scheduler:' }
class { 'airflow::service::worker:' }
- Example: Defining ldap authentication and mesos settings in hiera.
airflow::ldap_settings:
ldap_url: ldap:://<your.ldap.server>:<port>
user_filter: objectClass=*
user_name_attr: uid
bind_user: cn=Manager,dc=example,dc=com
bind_password: insecure
basedn: dc=example,dc=com
airflow::mesos_settings:
master: localhost:5050
framework_name: Airflow
task_cpu: 1
task_memory: 256
checkpoint: false
failover_timeout: 604800
authenticate: false
default_principal: admin
default_secret: admin
airflow
- Installs and configures airflow.airflow::service::worker
- Handles airflow's worker service.airflow::service::scheduler
- Handles airflow's scheduler service.airflow::service::webserver
- Handles airflow's webserver service.airflow::service::flower
- Handles airflow's flower service.
airflow::install
- Installs airflow python package.airflow::config
- Configures airflow.
- Fork the repository on Github
- Create a named feature branch (like
add_component_x
) - Commit your changes.
- Submit a Pull Request using Github