Enforce Best Practices for all your Airflow DAGs. ⭐
- Use function-level imports instead of top-level imports12 (see Top level Python Code)
- Use jinja macro instead of
Variable.get
(see Airflow Variables)
based on official Best Practices
To install it from PyPI run:
pip install airflint
NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way
UseJinjaVariableGet
rule can detect alltemplate_fields
and airflint works as expected.
Then just call it like this:
Alternatively you can add the following repo to your pre-commit-config.yaml
:
- repo: https://github.com/feluelle/airflint
rev: v0.2.0-alpha
hooks:
- id: airflint
args: ["-a"] # Use -a for replacing inplace
additional_dependencies: # Add all package dependencies you have in your dags, preferable with version spec
- apache-airflow
- apache-airflow-providers-cncf-kubernetes
To complete the UseFunctionlevelImports
rule, please add the autoflake
hook after the airflint
hook, as below:
- repo: https://github.com/pycqa/autoflake
rev: v1.4
hooks:
- id: autoflake
args: ["--remove-all-unused-imports", "--in-place"]
This will remove unused imports.
Footnotes
-
There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule. ↩
-
To remove top-level imports after running
UseFunctionLevelImports
rule, use a tool such as autoflake. ↩