Skip to content

Instantly share code, notes, and snippets.

@a1d4r

a1d4r/Dockerfile

Created February 15, 2024 13:57
Show Gist options
  • Save a1d4r/fdcf15ba680f60f87cbd16c7c3c92442 to your computer and use it in GitHub Desktop.
Save a1d4r/fdcf15ba680f60f87cbd16c7c3c92442 to your computer and use it in GitHub Desktop.
Download ZIP
Dockerfile + docker-compose.yml for development
Raw
README.md

my_awesome_app

NDA...

Installation

Local

  1. Create env file:
cp envs/.env.example envs/.env
  1. If you use pyenv, create and activate environment. You can read this article to get familiar with pyenv. Or you can just omit this step, and poetry will install venv for you.
pyenv install 3.11
pyenv virtualenv 3.11 my_awesome_app
pyenv local my_awesome_app
  1. If you don't have Poetry installed run:
make poetry-download
  1. Initialize poetry and install pre-commit hooks:
make install
make pre-commit-install
  1. Run formatters, and linters. Make sure there is no errors.
make format lint
  1. Run supporting services
docker compose --profile infra up -d
  1. Run the API
make api
  1. Run the task worker
make worker

You can open:

To stop all services run:

docker compose --profile dev --profile test down

Docker

  1. Create env file:
cp envs/.env.docker.example envs/.env.docker
  1. Build an image:
docker compose --profile dev build
  1. Run the app:
docker compose --profile dev up -d
  1. Run formatters and linters. Make sure there is no errors.
docker compose exec web make format lint

You can open:

To stop all services run:

docker compose --profile dev --profile test down

Tests

Local

Run supporting services for testing:

docker compose --profile test-infra up -d

Run tests:

make test

Stop all services:

docker compose --profile test-infra down

Docker

docker compose --profile test build
docker compose run test-runner

As simple as that

Migrations

Create migration:

alembic revision --autogenerate -m "Message"

Apply migrations:

alembic upgrade head

Revert last migration:

alembic downgrade -1

Makefile usage

Makefile contains a lot of functions for faster development.

1. Download and remove Poetry

To download and install Poetry run:

make poetry-download

To uninstall

make poetry-remove

2. Install all dependencies and pre-commit hooks

Install requirements:

make install

Pre-commit hooks could be installed after git init via

make pre-commit-install

3. Codestyle

Automatic formatting uses ruff.

make codestyle

# or use synonym
make format

Codestyle checks only, without rewriting files:

make check-codestyle

Update all dev libraries to the latest version using one comand

make update-dev-deps

4. Code security 

make check-security

This command identifies security issues with Safety and Bandit.

make check-security

To validate pyproject.toml use

make check-poetry

5. Linting and type checks

Run static linting with pylint and mypy:

make static-lint

6. Tests with coverage

Run pytest

make test

7. All linters

Of course there is a command to rule run all linters in one:

make lint

the same as:

make test && make check-codestyle && make static-lint && make check-safety

8. Docker 

make docker-build

which is equivalent to:

make docker-build VERSION=latest

Remove docker image with

make docker-remove

More information about docker.

9. Cleanup

Delete pycache files 

make pycache-remove

Remove package build

make build-remove

Delete .DS_STORE files

make dsstore-remove

Remove .mypycache

make mypycache-remove

Or to remove all above run:

make cleanup

Credits

This project was generated with python-package-template

Raw
docker-compose.yml
version: "3.9"
services:
web:
build:
context: .
dockerfile: Dockerfile
target: development
command: bash -c "alembic upgrade head && uvicorn my_awesome_app.app:app --host 0.0.0.0 --port 8000 --reload"
env_file:
- ./envs/.env.docker
ports:
- "8000:8000"
volumes:
- .:/app
depends_on:
postgres:
condition: service_healthy
profiles:
- dev
worker:
build:
context: .
dockerfile: Dockerfile
target: development
command: taskiq worker my_awesome_app.broker:broker my_awesome_app.tasks --reload
env_file:
- ./envs/.env.docker
volumes:
- .:/app
depends_on:
postgres:
condition: service_healthy
profiles:
- dev
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: user
POSTGRES_PASSWORD: password
POSTGRES_DB: my_awesome_app
PGDATA: /data/postgres
ports:
- "5432:5432"
volumes:
- postgres-data:/data/postgres
restart: unless-stopped
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U user -d my_awesome_app" ]
interval: 5s
timeout: 5s
retries: 5
profiles:
- dev
- infra
minio:
image: minio/minio
ports:
- "9000:9000"
- "9001:9001"
volumes:
- minio-data:/data
environment:
MINIO_ROOT_USER: user
MINIO_ROOT_PASSWORD: password
command: server --console-address ":9001" /data
profiles:
- dev
- infra
redis:
image: redis:7-alpine
ports:
- "6379:6379"
command: redis-server --save 60 1 --loglevel warning
volumes:
- redis-data:/data
restart: unless-stopped
profiles:
- dev
- infra
rabbitmq:
image: rabbitmq:3-management-alpine
ports:
- "5672:5672"
- "15672:15672"
restart: unless-stopped
profiles:
- dev
- infra
createbuckets:
image: minio/mc
depends_on:
- minio
entrypoint: >
/bin/sh -c "
/usr/bin/mc alias set myminio http://minio:9000 user password;
/usr/bin/mc mb myminio/my_awesome_app;
/usr/bin/mc anonymous set public myminio/my_awesome_app;
exit 0;
"
profiles:
- dev
- infra
test-postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: user
POSTGRES_PASSWORD: password
POSTGRES_DB: postgres
ports:
- "5433:5432"
restart: unless-stopped
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U user -d test" ]
interval: 5s
timeout: 5s
retries: 5
profiles:
- test
- test-infra
test-minio:
image: minio/minio
ports:
- "9002:9000"
- "9003:9003"
environment:
MINIO_ROOT_USER: user
MINIO_ROOT_PASSWORD: password
command: server --console-address ":9003" /data
profiles:
- test
- test-infra
test-runner:
build:
context: .
dockerfile: Dockerfile
target: development
command: bash -c "alembic upgrade head && make test"
env_file:
- ./envs/.env.docker.test
volumes:
- .:/app
depends_on:
test-postgres:
condition: service_healthy
test-minio:
condition: service_started
profiles:
- test
volumes:
postgres-data:
minio-data:
redis-data:
Raw
Dockerfile
# Source: https://gist.github.com/usr-ein/c42d98abca3cb4632ab0c2c6aff8c88a
################################
# PYTHON-BASE
# Sets up all our shared environment variables
################################
FROM python:3.11-slim as python-base
# python
ENV PYTHONUNBUFFERED=1 \
# prevents python creating .pyc files
PYTHONDONTWRITEBYTECODE=1 \
\
# pip
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_DEFAULT_TIMEOUT=100 \
\
# poetry
# https://python-poetry.org/docs/configuration/#using-environment-variables
POETRY_VERSION=1.6.1 \
# make poetry install to this location
POETRY_HOME="/opt/poetry" \
# make poetry create the virtual environment in the project's root
# it gets named `.venv`
POETRY_VIRTUALENVS_IN_PROJECT=true \
# do not ask any interactive question
POETRY_NO_INTERACTION=1 \
\
# paths
# this is where our requirements + virtual environment will live
PYSETUP_PATH="/opt/pysetup" \
VENV_PATH="/opt/pysetup/.venv"
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
# deps for installing poetry
curl \
# deps for building python deps
build-essential
# prepend poetry and venv to path
ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"
################################
# BUILDER-BASE
# Used to build deps + create our virtual environment
################################
FROM python-base as builder-base
# install poetry - respects $POETRY_VERSION & $POETRY_HOME
# The --mount will mount the buildx cache directory to where
# Poetry and Pip store their cache so that they can re-use it
RUN --mount=type=cache,target=/root/.cache \
curl -sSL https://install.python-poetry.org | python3 -
# copy project requirement files here to ensure they will be cached.
WORKDIR $PYSETUP_PATH
COPY poetry.lock pyproject.toml ./
# install runtime deps - uses $POETRY_VIRTUALENVS_IN_PROJECT internally
RUN --mount=type=cache,target=/root/.cache \
poetry install --without=dev
################################
# DEVELOPMENT
# Image used during development / testing
################################
FROM python-base as development
WORKDIR $PYSETUP_PATH
# copy in our built poetry + venv
COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH
# quicker install as runtime deps are already installed
RUN --mount=type=cache,target=/root/.cache \
poetry install --with=dev
# will become mountpoint of our code
WORKDIR /app
CMD ["uvicorn", "my_awesome_app.app:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
################################
# PRODUCTION
# Final image used for runtime
################################
FROM python-base as production
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH
COPY ./my_awesome_app /app/my_awesome_app
COPY ./migrations /app/migrations
COPY ./alembic.ini /app/alembic.ini
WORKDIR /app
CMD ["uvicorn", "my_awesome_app.app:app", "--host", "0.0.0.0", "--port", "8000"]
Raw
Makefile
#* Variables
SHELL := /usr/bin/env bash
PYTHON := python3
#* Docker variables
IMAGE := my_awesome_app
VERSION := latest
#* Directories with source code
CODE = my_awesome_app tests
CODE_FORMAT = my_awesome_app tests migrations
TESTS = tests
#* Application
api:
uvicorn my_awesome_app.app:app --reload
worker:
taskiq worker my_awesome_app.broker:broker my_awesome_app.tasks --reload
#* Poetry
.PHONY: poetry-download
poetry-download:
curl -sSL https://install.python-poetry.org | $(PYTHON) -
.PHONY: poetry-remove
poetry-remove:
curl -sSL https://install.python-poetry.org | $(PYTHON) - --uninstall
#* Installation
.PHONY: install
install:
poetry lock --no-interaction --no-update
poetry install --no-interaction
.PHONY: pre-commit-install
pre-commit-install:
poetry run pre-commit install
#* Formatters
.PHONY: codestyle
codestyle:
poetry run ruff format $(CODE_FORMAT)
poetry run ruff check $(CODE) --fix-only
.PHONY: format
format: codestyle
#* Test
.PHONY: test
test:
poetry run pytest -n auto --cov $(TEST_FILTER)
poetry run coverage xml
# Validate pyproject.toml
.PHONY: check-poetry
check-poetry:
poetry check
.PHONY: check-codestyle
check-codestyle:
poetry run ruff format $(CODE_FORMAT) --check
#* Static linters
.PHONY: check-ruff
check-ruff:
poetry run ruff check $(CODE) --no-fix
.PHONY: check-mypy
check-mypy:
poetry run mypy --install-types --non-interactive --config-file pyproject.toml $(CODE)
.PHONY: static-lint
static-lint: check-ruff check-mypy
#* Check safety
.PHONY: check-safety
check-safety:
poetry run safety check --full-report
.PHONY: lint
lint: check-poetry check-codestyle static-lint
.PHONY: update-dev-deps
update-dev-deps:
poetry add -G dev mypy@latest pre-commit@latest pytest@latest \
coverage@latest safety@latest typeguard@latest ruff@latest
#* Docker
# Example: make docker-build VERSION=latest
# Example: make docker-build IMAGE=some_name VERSION=0.1.0
.PHONY: docker-build
docker-build:
@echo Building docker $(IMAGE):$(VERSION) ...
docker build -t $(IMAGE):$(VERSION) .
# Example: make docker-remove VERSION=latest
# Example: make docker-remove IMAGE=some_name VERSION=0.1.0
.PHONY: docker-remove
docker-remove:
@echo Removing docker $(IMAGE):$(VERSION) ...
docker rmi -f $(IMAGE):$(VERSION)
.PHONY: docker-up
docker-up:
docker compose up
#* Cleaning
.PHONY: pycache-remove
pycache-remove:
find . | grep -E "(__pycache__|\.pyc|\.pyo$$)" | xargs rm -rf
.PHONY: dsstore-remove
dsstore-remove:
find . | grep -E ".DS_Store" | xargs rm -rf
.PHONY: mypycache-remove
mypycache-remove:
find . | grep -E ".mypy_cache" | xargs rm -rf
.PHONY: ipynbcheckpoints-remove
ipynbcheckpoints-remove:
find . | grep -E ".ipynb_checkpoints" | xargs rm -rf
.PHONY: pytestcache-remove
pytestcache-remove:
find . | grep -E ".pytest_cache" | xargs rm -rf
.PHONY: ruffcache-remove
ruffcache-remove:
find . | grep -E ".ruff_cache" | xargs rm -rf
.PHONY: build-remove
build-remove:
rm -rf build/
.PHONY: reports-remove
reports-remove:
rm -rf reports/
.PHONY: cleanup
cleanup: pycache-remove dsstore-remove mypycache-remove ruffcache-remove \
ipynbcheckpoints-remove pytestcache-remove reports-remove
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment