Skip to content

Enforce Best Practices for all your Airflow DAGs. ⭐

License

Notifications You must be signed in to change notification settings

feluelle/airflint

Repository files navigation

airflint

pre-commit.ci status test workflow codeql-analysis workflow codecov PyPI version License PyPI - Python Version PyPI version

Enforce Best Practices for all your Airflow DAGs. ⭐

⚠️ airflint is still in alpha stage and has not been tested with real world Airflow DAGs. Please report any issues you face via GitHub Issues, thank you. 🙏

🧑‍🏫 Rules

based on official Best Practices

Requirements

airflint is tested with:

Main version (dev) Released version (0.3.2-alpha)
Python 3.9, 3.10, 3.11.0-alpha - 3.11.0 3.9, 3.10
Apache Airflow >= 2.0.0 >= 2.0.0

🚀 Get started

To install it from PyPI run:

pip install airflint

NOTE: It is recommended to install airflint into your existing airflow environment with all your providers included. This way UseJinjaVariableGet rule can detect all template_fields and airflint works as expected.

Then just call it like this:

usage

pre-commit

Alternatively you can add the following repo to your pre-commit-config.yaml:

  - repo: https://github.com/feluelle/airflint
    rev: v0.3.2-alpha
    hooks:
      - id: airflint
        args: ["-a"]  # Use -a to apply the suggestions
        additional_dependencies:  # Add all package dependencies you have in your dags, preferable with version spec
          - apache-airflow
          - apache-airflow-providers-cncf-kubernetes

To complete the UseFunctionlevelImports rule, please add the autoflake hook after the airflint hook, as below:

  - repo: https://github.com/pycqa/autoflake
    rev: v1.4
    hooks:
      - id: autoflake
        args: ["--remove-all-unused-imports", "--in-place"]

This will remove unused imports.

❤️ Contributing

I am looking for contributors who are interested in..

  • testing airflint with real world Airflow DAGs and reporting issues as soon as they face them
  • optimizing the ast traversing for existing rules
  • adding new rules based on best practices or bottlenecks you have experienced during Airflow DAGs authoring
  • documenting about what is being supported in particular by each rule
  • defining supported airflow versions i.e. some rules are bound to specific Airflow features and version

For questions, please don't hesitate to open a GitHub issue.

Footnotes

  1. There is a PEP for Lazy Imports targeted to arrive in Python 3.12 which would supersede this rule.

  2. To remove top-level imports after running UseFunctionLevelImports rule, use a tool such as autoflake.

About

Enforce Best Practices for all your Airflow DAGs. ⭐

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •  

Languages