Hynek Schlawack

Why I Still Use Python Virtual Environments in Docker

2024-09-02T11:00:00Z

Whenever I publish something about my Python Docker workflows, I invariably get challenged about whether it makes sense to use virtual environments in Docker containers. As always, it’s a trade-off, and I err on the side of standards and predictability.

Production-ready Python Docker Containers with uv

2024-08-28T11:00:00Z

Starting with 0.3.0, Astralâ€™s uv brought many great features, including support for cross-platform lock files uv.lock. Together with subsequent fixes, it has become Pythonâ€™s finest workflow tool for my (non-scientific) use cases. Hereâ€™s how I build production-ready containers, as fast as possible.

Python Project-Local Virtualenv Management Redux

2024-04-02T00:00:00Z

One of my first TIL entries was about how you can imitate Nodeâ€™s node_modules semantics in Python on UNIX-like operating systems. A lot has happened since then (to the better!) and itâ€™s time for an update. direnv still rocks, though.

Donâ€™t Start Pull Requests from Your Main Branch

2023-12-21T05:00:00Z

When contributing to other usersâ€™ repositories, always start a new branch in your fork.

Two Ways to Turbo-Charge tox

2023-06-26T14:00:00Z

No, it’s not (just) run-parallel â€“ let’s cut the local tox runtime by 75%!

Subclassing, Composition, Python, and You

2023-04-22T00:00:00Z

Ever seen a code base where understanding a simple method meant jumping through tangled class hierarchies? We all have! And while “Favor composition over inheritance!” is almost as old as object-oriented programming, strictly avoiding all types of subclassing leads to verbose, un-Pythonic code. So, what to do?

Why I Like Nox

2023-01-18T12:00:00Z

Ever since I got involved with open-source Python projects, tox has been vital for testing packages across Python versions (and other factors). However, lately, I’ve been increasingly using Nox for my projects instead. Since I’ve been asked why repeatedly, I’ll sum up my thoughts.

Surprising Consequences of macOSâ€™s Environment Variable Sanitization

2023-01-09T08:00:00Z

Why does DYLD_LIBRARY_PATH keep disappearing!?

How Iâ€™m a Productive Programmer With a Memory of a Fruit Fly

2022-09-19T00:00:00Z

A love letter to tools that changed everything for me.

Recursive Optional Dependencies in Python

2022-07-29T06:00:00Z

One of my (slowly evaporating) reasons why I like putting packaging metadata into an executable setup.py is the ability to have optional dependencies that are combinations of others. As of pip 21.2, this is possible without running code.

â€œDonâ€™t Mock What You Donâ€™t Ownâ€ in 5 Minutes

2022-06-21T09:00:00Z

A common issue when writing tests for real-world software is how to deal with third-party dependencies. Let’s examine an old, but counter-intuitive principle.

import attrs

2021-12-28T00:00:00Z

An attempt at catharsis. This is a deeply personal blog post about the most influential project I’ve ever created: attrs, the progenitor of modern Python class utilities. I’m retelling its history from my perspective, how it begot dataclasses, and how I’m leading it into the future.

How to Ditch Codecov for Python Projects

2021-11-18T08:00:00Z

Codecov’s unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn’t rely on third-party services.

Announcing a New Section: TIL

2021-08-25T12:00:00Z

Trying out something new: today I’m launching my own Today I Learned section. In this essay I will sum up what my plans and hopes are.

Subclassing in Python Redux

2021-06-22T15:00:00Z

The conflict between subclassing and composition is as old as object-oriented programming. The latest crop of languages like Go or Rust prove that you donâ€™t need subclassing to successfully write code. But whatâ€™s a pragmatic approach to subclassing in Python, specifically?

Canonical SEO Failure

2021-04-28T00:00:00Z

This post is less about teaching and more about schadenfreude amusement for you, and catharsis for me. Itâ€™s the story of how one unfortunate HTML tag kicked me off almost all search engines and my months-long way back. And why it didn’t matter in the end.

Semantic Versioning Will Not Save You

2021-03-02T00:00:00Z

The widely used Python package cryptography changed their build system to use Rust for low-level code, which caused an emotional GitHub thread. Enthusiasts of 32-bit hardware from the 1990s aside, a vocal faction stipulated adherence to Semantic Versioning from the maintainers, claiming it wouldâ€™ve prevented all grief. I will show you not only why this is wrong but also how relying on Semantic Versioning hurts you â€“ the user.

Lie vs Lay

2021-02-03T00:00:00Z

Lie and lay are infamously confusing to non-native speakers. Itâ€™s so bad that it sparked a cottage industry of click-baity articles full of sketchy ads. Since English is my third language, I stumbled a lot myself until I wrote this cheatsheet.

Waiting in asyncio

2020-05-21T00:00:00Z

One of the main appeals of using Pythonâ€™s asyncio is being able to fire off many coroutines and run them concurrently. How many ways do you know for waiting for their results?

Classy Abstractions

2020-05-18T00:00:00Z

Let’s talk about abstractions and Pythonic code.

Why You Should Document Your Tests

2020-05-04T00:00:00Z

Some projects have the policy that all tests must have an explanatory comment â€“ including all of mine. At first, I found that baffling. If that’s you right now, this article is for you.

Python in GitHub Actions

2020-03-09T00:00:00Z

GitHub’s own CI called GitHub Actions has been out of closed beta for a while and offers generous free quotas and a seamless integration with the rest of the site. Let’s have a look at how to use it for an open source Python package.

Python in Production

2020-02-23T16:45:00Z

I’m missing a key part from the public Python discourse and I would like to help to change that.

Python Packaging Metadata

2020-02-14T00:00:00Z

Since this topic keeps coming up, Iâ€™d like to briefly share my thoughts on Python package metadata because itâ€™s â€“ as always â€“ more complex than it seems.

Books That Changed My Life

2019-09-23T00:00:00Z

People always ask for book recommendations, so here’s my list of books that changed my life in one way or another. None of them is related to tech or programming, but all affect how I think and work.

Python in Azure Pipelines, Step by Step

2019-06-03T09:14:00Z

Since the acquisition of Travis CI, the future of their free offering is unclear. Azure Pipelines has a generous free tier, but the examples I found are discouragingly complex and take advantage of features like templating that most projects don’t need. To close that gap, this article shows you how to move a Python project with simple CI needs from Travis CI to Azure Pipelines.

The Price of the Hallway Track

2019-05-15T18:00:00Z

There are many good reasons not to go to every talk possible when attending conferences. However, increasingly, it became hip to boast about avoiding going to talks â€“ encouraging others to follow suit. That rubs me the wrong way as a speaker, and I’ll try to explain why.

Maintaining a Python Project When Itâ€™s Not Your Job

2019-05-03T00:00:00Z

PyPI is a gold mine of great packages but those packages have to be written first. More often than not, projects that millions of people depend on are written and maintained by only one person. If youâ€™re unlucky, that person is you! This talk tries to lighten the burden by giving you useful tools and approaches.

Python Application Dependency Management

2018-11-29T17:00:00Z

We have more ways to manage dependencies in Python applications than ever. But how do they fare in production? Unfortunately this topic turned out to be quite polarizing and was at the center of a lot of heated debates. This is my attempt at an opinionated review through a DevOps lens.

How to Write Deployment-friendly Applications

2018-05-12T00:00:00Z

The DevOps movement gave us many ways to put Python applications into production. But how can you practically structure and configure your applications to make them indifferent to the environment they run in? How do secrets fit into the picture? And where do you put that log file?

Python Hashes and Equality

2017-11-20T06:45:00Z

Most Python programmers donâ€™t spend a lot of time thinking about how equality and hashing works. It usually just works. However thereâ€™s quite a bit of gotchas and edge cases that can lead to subtle and frustrating bugs once one starts to customize their behavior â€“ especially if the rules on how they interact arenâ€™t understood.

Why Your Dockerized Application Isnâ€™t Receiving Signals

2017-06-19T00:00:00Z

Proper cleanup when terminating your application isnâ€™t less important when itâ€™s running inside of a Docker container. Although it only comes down to making sure signals reach your application and handling them, thereâ€™s a bunch of things that can go wrong.

On Conference Speaking

2017-05-30T00:00:00Z

Iâ€™ve seen quite a bit of the world thanks to being invited to speak at conferences. Since some people are under the impression that serial conference speakers possess some special talents, Iâ€™d like to demystify my process by walking you through my latest talk from start to finish.

Solid Snakes or: How to Take 5 Weeks of Vacation

2017-05-19T00:00:00Z

No matter whether you run a web app, search for gravitational waves, or maintain a backup script: reliability of your systems make the difference between sweet dreams and production nightmares at 4am.

Better Python Object Serialization

2016-08-22T12:30:00Z

The Python standard library is full of underappreciated gems. One of them allows for simple and elegant function dispatching based on argument types. This makes it perfect for serialization of arbitrary objects â€“ for example to JSON in web APIs and structured logs.

Please Fix Your Decorators

2016-08-01T00:00:00Z

If your Python decorator unintentionally changes the signatures of my callables or doesnâ€™t work with class methods, itâ€™s broken and should be fixed. Sadly most decorators are broken because the web is full of bad advice.

Get Instrumented: How Prometheus Can Unify Your Metrics

2016-05-31T00:00:00Z

To get real time insight into running applications you need to instrument them and collect metrics: count events, measure times, expose numbers. That used to be a clusterfuck of technologies and approaches. Prometheus changes that.

Conditional Python Dependencies

2016-05-19T00:00:00Z

Since the inception of wheels that install Python packages without executing arbitrary code, we need a static way to encode conditional dependencies for our packages. Thanks to PEP 508 we do have a blessed way but sadly the prevalence of old setuptools and pip versions make it a minefield to use.

Python 3 in 2016

2016-02-17T00:00:00Z

My completely anecdotal view on the state of Python 3 in 2016. Based on my own recent experience, observations, and exchanges with other members of the Python community.

hasattr() â€“ A Dangerous Misnomer

2016-01-13T00:00:00Z

Donâ€™t use Pythonâ€™s hasattr() unless youâ€™re writing Python 3-only code and understand how it works.

Storing Passwords in a Highly Parallelized World

2016-01-06T00:00:00Z

Why â€œUse bcrypt.â€ is not the best recommendation (anymore).

Testing & Packaging

2015-10-19T00:00:00Z

How to ensure that your tests run code that you think they are running, and how to measure your coverage over multiple tox runs (in parallel!).

Beyond grep: Practical Logging and Metrics

2015-04-12T00:00:00Z

Your Python applications are running but youâ€™re wondering what they are doing? The only clue about their current state is the server load after ssh-ing into the servers? Letâ€™s change that!

virtualenv Lives!

2014-09-15T15:00:00Z

Setting up Python to the point to be able install packages from PyPI can be annoying and time-intensive. Even worse are OS-provided installations that start throwing cryptic error messages. Especially desktops are prone to that but itâ€™s possible to break the whole toolchain of a server by installing some shiny package you heard about on reddit.

The Sorry State Of SSL

2014-04-12T00:00:00Z

TLS is the best technology we have for securing our communications. It comes with many sharp edges though. This talk tries to jumpstart a rough understanding and these links should help you to complete the picture.

Apple OpenSSL Verification Surprises

2014-03-03T15:30:00Z

Apple ships a patched version of OpenSSL with macOS. If no precautions are taken, their changes rob you of the power to choose your trusted certificate authorities (CAs) and break the semantics of a callback that can be used for custom checks and verifications in client software.

Sharing Your Labor of Love: PyPI Quick and Dirty

2013-07-29T00:00:00Z

A completely incomplete guide to packaging a Python module and sharing it with the world on PyPI.

Life after Google Reader

2013-06-27T00:00:00Z

Google killed its Reader and my beloved Reeder for Mac and iPad officially wonâ€™t get updated in time. I think to have found an adequate setup to replace both.

A Short Summary on Sybase SQL Anywhere and Python

2013-05-29T00:00:00Z

As some of my older rage-filled articles indicated, weâ€™re still running some services on SAP’s SQL Anywhere. Since it cost me many hours and sanity wrangling, I think it may be helpful to others to summarize the current situation for Python developers.

Know Your Models

2013-05-15T00:00:00Z

In web development, we have an unfortunate double meaning for the word models. As evident as the separation of those two seems to seasoned developers, it shows again and again that itâ€™s not as apparent to beginners.

Solid Python Deployments for Everybody

2013-03-16T00:00:00Z

Without orientation, deployments of Python applications can be tiresome and even painful. This talk attempts to replace anxiety and pain through informed annoyance.

Taking Some Pain out of Python Logging

2013-03-06T00:00:00Z

Even the best of us hate logging in Python sometimes. And while a lot of its problems are actually just bad docs and terrible defaults in the past, there is some pain that can be avoided.

Hardening Your Web Serverâ€™s SSL Ciphers

2013-02-05T00:00:00Z

There are many wordy articles on configuring your web serverâ€™s TLS ciphers. This is not one of them. Instead, I will share a configuration that scores a straight â€œAâ€ on Qualysâ€™s SSL Server Test in 2023.

Using Celery with Pyramid

2012-07-20T00:00:00Z

This one falls under: â€œI knew there has to be an easy way!â€

How I Stopped Worrying and Started Loving PyLadies

2012-07-13T00:00:00Z

When I read about PyLadies for the first time, my thoughts were a common knee-jerk: â€œseparation is bad, dividing the community, â€¦â€. Like many of my privileged peers, I was pro-diversity but I thought this is the wrong way. My views changed over time and I filed it under â€œlessons learnedâ€. Unfortunately, my old thinking patterns donâ€™t cease to pop up in discussions, so I decided to share my perspective.

My Road to the Python Commit Bit

2012-05-19T00:00:00Z

Like many OSS fans, I always wanted to be an active part of the movement. My last big project was for the Amiga in the past millennium though. Nowadays Iâ€™m happy that after years of small-scale dabbling on various projects Iâ€™ve found my haven. Iâ€™d like to share my way to my recent gain of push privileges on the Python project and hope to inspire some of you to do the same.

Python Application Deployment with Native Packages

2012-05-03T00:00:00Z

Speed, reproducibility, easy rollbacks, and predictability is what we strive for when deploying our diverse Python applications. And thatâ€™s what we achieved by leveraging virtual environments and Linux system packages.

Python Deployment Anti-Patterns

2012-04-23T00:00:00Z

Deploying web applications is hard. No shiny continuous deployment talk and no DevOps coolness can change that. Or to use DevOps Boratâ€™s words: â€œIs all fun and game until you are need of put it in production.â€œ There are some mistakes I see people doing again and again so Iâ€™d like to address them here.

Celery and Sybase SQL Anywhere

2012-02-25T00:00:00Z

In our newest installation of â€œwhy you should not use Sybase SQL Anywhereâ€ Iâ€™d like to report the latest problem I had to solve: for some reason, I couldnâ€™t connect using sqlanydb from Celery tasks.

Fleeing from Gmail

2012-01-26T00:00:00Z

So you came to the same conclusion as I: Google is actually evil indeed. That makes it kind of uncomfortable to have all your emails over there, doesnâ€™t it? I for one decided that itâ€™s time to leave and will show you how to do the same using an UNIX based OS.

My 2011

2011-12-31T00:00:00Z

I never did a retrospective but 2011 deserves one.

Dead Simple Connection Pooling with Twisted

2011-12-14T00:00:00Z

There is this common notion, that asynchronous IO is hard and that writing a custom connection pool is even harder. The nice thing however is, that in reality asynchronous IO is just â€œweirdâ€ in the beginning â€“ and that a connection pool using async IO is so simple it hurts.

MacVim and the Clipboard

2011-12-06T00:00:00Z

After switching to the Mac, I had one big itch that spoiled all the bliss: MacVim sometimes simply refused to cooperate with the system clipboard. As you can imagine, an editor that canâ€™t exchange text with other software is a rather painful thing.

Why Tomatoes Are Awesome

2011-10-01T00:00:00Z

No, this isnâ€™t going to be a smug post about nutrition. These tomatoes Iâ€™m going to talk about arenâ€™t for eating butÂ for kicking my ass to be more productive. Iâ€™m talking about the Pomodoro Technique of course.

My mutt and Gmail Setup

2011-08-22T00:00:00Z

So you want quick offline access to your mails using mutt to fully exploit your SSD and yet still have everything nicely working in Googleâ€™s web interface? Additionally, you would like to have access to your Google Contacts just like in your phone and everywhere else? Iâ€™ll show you how!

Solarized â€“ Love on First Sight

2011-07-10T00:00:00Z

Thereâ€™s one thing hackers are opinionated about as about the right editor: The right color scheme.

Mercurial to Git

2011-05-10T00:00:00Z

Mostly a note to myself as I forget it regularly.

Twisted Sybase SQL Anywhere

2011-04-27T00:00:00Z

Using the official sqlanydb driver for Python together with Twistedâ€™s adbapi produces not-so-occasional crashes as of today (sqlanydb 1.0.2, Twisted 11.0.0). Apparently, the official SQL Anywhere drivers arenâ€™t thread-safe. It cost me several days to figure out because I was searching the fault in my code so I hope to spare you some pain.

Django & Postgres & SSL

2011-04-12T00:00:00Z

Iâ€™d thought something like this is a FAQ but the database docs on postgres donâ€™t write a bit about forcing Django to connect using SSL to the database server.

Information Diet

2011-04-01T00:00:00Z

Be careful to not overeat.

Django and Remote Sybase Servers

2011-02-07T00:00:00Z

It took me a while to figure it out, so I decided to share.

Freezing Kindle

2010-12-07T00:00:00Z

After a few months of use my Kindle started to freeze randomly. At first I blamed the cold as it happened to be winter in that moment. Fortunately, it turned out to be something different.

LDAP: A Gentle Introduction

2007-02-13T00:00:00Z

The perception of LDAP (Lightweight Directory Access Protocol) is ambivalent. On the one hand, it is widely supported as a common authentication backend. On the other hand, there’s very little and poor documentation mainly targeted toward a particular case (for example, replacing NIS with LDAP).

MySQL Replication

2007-02-09T00:00:00Z

Sounds harder than it is â€“ especially when reading the official docs. But if you want to synchronize two DBs, just tell the â€œmasterâ€ to write a log and slave to read it.

PCAP Format for Logs

2005-11-22T00:00:00Z

While developing a network sniffer I had to find a way to write pcap logs. However the docs I found were rather fragmented. I try to do a short roundup here. In fact, the format is pretty plain and itâ€™s a pity that there seems not to be a quickâ€™nâ€™easy doc for it.