Skip to content

"Main" tip caching of CI image is broken #42999

Closed
@potiuk

Description

@potiuk

It looks like after the provider move in #42505, "main" tip caching for airflow is broken. Before the change installing airflow had use two stages caching:

  1. Airflow has been installed from main first (which allowed to perform pre-installation without invalidation of docker layer)
  2. Then additional installation was done after copying latest sources and pyproject.toml/hatch

This allowed for very fast rebuilds of the image using remote cache built on ghcr.io, when only source files or dependencies of airlfow changed.

It usually took 2-5 seconds to rebuild such image. Currently it looks like second installation step is running again full installation of airlfow (it takes about 2 minutes), which means that likely some mechanism of caching is broken.

Image

We should review and updated Dockerfile.ci and scripts to bring the speed back.

Metadata

Metadata

Assignees

Labels

area:CIAirflow's tests and continious integration

Type

No type

Projects

  • Status

    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions