Clickbaity title, esp. when I think the article establishes that uv/pixi are... good? Using those, is it really so bad compared to other commonly used languages?
(Also, I liked learning about pixi; I should learn about the Conda ecosystem at some point. Although at this point, uv has solved my last remaining niggles with Poetry. I understand the more science-y Python projects might not be solvable by uv, but mine are.)
Using those, is it really so bad compared to other commonly used languages?
As someone with a non-dev team, getting people to configure their shell to disable the bad defaults in uv (like downloading Python interpreters from the Internet), and using uv consistently instead of pip, yeah, it's pretty bad.
I was very deliberate with "compared to other commonly used languages"; in the end, there's rbenv, nvm, rustup, ... For some time, we've been living in a time when questions like "what is a Linux distribution" are undergoing changes. Frankly, I have no idea how things will play out in the next years, but in the meantime in most of the scenarios I deal with currently, I cannot force Linux distribution uniformity and stuff like containers is undesirable, so rustup/uv are what's most productive to me. Of course, in many scenarios I don't deal with, it's not a good idea.
I do agree getting people to use these tools consistently is hard. In my opinion, most people feel these things are an obstacle, and want to spend as little time as possible on it. In a way, they are right! We are not paid to have smooth environments. Unfortunately, I think we people are bad at estimating how much time we spend due to non-smooth environments.
having used python and “anything else”, yes. in part this is due to the tasks one performs with python - a java project can just happily download pure-java dependencies from maven because it doesn’t need CUDA. still, the end-user experience is the disproportionately bad relative to the language’s mainstream nature.
Well, I think Java is the outlier here. It's awesome that pure-Java dependencies are the norm. Also Java is pretty good with compatibility between versions. And Maven, as controversial as it is, has many accidental benefits. Java has a terrible reputation, but it's not one of the most widespread languages gratuitously.
But still, since around the time that Poetry (and others) appeared, I feel at least there's a couple of decent/acceptable options. Still, you will find horribly broken environments out there, and it's a headache explaining all that is needed to use one of those decent/acceptable options,but there aren't that many better platforms. (Yes, Cargo is awesome. It also was born much later than Python and it learned a lot of lessons from what came before.)
The author of this article seems to put a strong emphasis on (in his opinion) python dependency manager shouldn't be written in python. This in my opinion is a bit misguided, if a python developer wants to fix a bug or contribute a feature if its written in python they can but if its written in another language like uv is written in Rust you now have to learn Rust. Quite honestly what language it's written in shouldn't even be a criteria, its about the tool doing its job.
His comments about nix shows he really didn't use it much before forming an opinion. Which is fine.
All in all, it's a sales pitch to use uv as opposed to balanced review of all the options, but I do agree that python dependency management can be a lot better and it's quirky.
I suppose it's a matter of opinion as to whether the reviews are "balanced", but the author did include a lot of information about the existing tools. He's right, too: uv is significantly better on all the objective metrics.
I've used Nix and formed an opinion about it: it's just not an appropriate solution for the Python distribution scenarios I've worked with. I'm a little surprised it's even included in the post.
I generally have to agree. If Python had a uv competitor written in PyPy's RPython and translated to a self-contained native binary, then I could see that... but dependency-managing a scripting language in itself is inherently problematic.
This is definitely a topic that inspires a lot of strong opinions, but I have trouble reading the same conclusion as the author. I suspect part of this is there are different audiences for Python and they have wildly different expectations for dependencies.
You use Python for ML/AI: You have a million packages, Python is basically the "glue" that sticks your packages together and you should probably use ul or frankly bite the bullet and just switch to only using containers for local and remote work.
You use Python for web stuff. You have "some" packages but this list doesn't change a lot and the dependencies don't have an extremely complex relationship with each other. When you need to do something you should probably attempt to do it with the standard library and only if absolutely necessary go get another package.
You use Python for system administration stuff. Your life is pretty much pure pain. You don't understand virtual environments, you have trouble understanding pip vs using a Linux package manager to go get your dependency, you don't have a clean environment to work from ever. You should probably switch to a different language.
As someone who uses Python primarily for 2, I'm just not seeing anywhere near as much pain as people are talking about. venv and pip-tools solves most of my problems and for work stuff I just use containers because that's how the end product is going to be consumed anyway so why bother with a middle-state.
In my experience this actually works pretty well, as long as you never touch a Python package manager. Important Python libraries do tend to be available as distribution packages, a README with "apt install py3-foo py3-bar" isn't complicated, and you can make your Python into a proper distribution package if you want automatic dependency management. "System administration tasks" tend not to require a gazillion libraries, nor tracking the very latest version of your libraries.
Mixing distribution packages with pip, poetry, uv, ... is every bit as painful as you describe, though - I agree that one should avoid that if at all possible!
So what you are describing is how people did it for a long time. You write a python script, you install the Debian package for that script system-wide and you bypass the entire pip ecosystem.
My understanding from Debian maintainers and from Python folks is this is canonically not the right way to do it anymore. If your thing has Python dependencies you should make a virtual environment, install those dependencies inside of that environment. Like if you want to ship Python as a .deb I think you need to end up doing something like this:
To be honest, that's one of the big reasons I use Rust for "shell scripting" where I used to use Python. Deployment and maintenance across distro release upgrades are just so much easier with a static binary than with a venv.
I can only imagine that's also why so many people are now using Go for that sort of thing.
(To be honest, I'm hoping that going from Kubuntu 22.04 LTS to 24.04 LTS on this machine will be much less irritating than previous version bumps because of how much I've moved from Python to Rust.)
Yeah, used Python professionally for a decade. Never had a single problem. Came to think that people railing against Python dependency management had a "them problem".
Python packaging has its issues but have you seen Node where there are a handful of wildly incompatible ways to build/include libraries and nobody can push anything forward?
You use Python for system administration stuff. Your life is pretty much pure pain. You don’t understand virtual environments, you have trouble understanding pip vs using a Linux package manager to go get your dependency, you don’t have a clean environment to work from ever. You should probably switch to a different language.
I have been finding success with uv run for these users and either uploading packages to the registry for them to execute with uvx or inline script metadata for dependencies.
I love being able to tell somebody to just go install pipx and then pipx install a tool directly from a git repo. It's so straightforward! I'm very glad that uv supports this too. It's quickly becoming the only python dependency tool I need.
It is a nightmare. Venv make no sense and I have yet to understand how to use pip-tools. That is after having written and fixed package managers in othet languages and even being a nixpkgs maintainer.
Even nix is easier to write and use that venv. As long as venv are part of the UX of your solution, this is not going to fly.
Can you clarify how venvs make no sense to you? They seem to be the one part of Python that actually make sense to me, and bring some sanity. (I still prefer Nix, too.)
I'm surprised? This isn't me attacking you, just genuine curiosity. My primary issue with onboarding interns or junior folks onto python projects has been explaining that venv exists, but once they grasp the basics of it seems to "just work". Especially with pip-compile and pyproject.toml there doesn't seem to be a ton of places for it to go wrong, but I'd love to know what about it doesn't work for you.
In my experience, case 1 usually results in "it works on my machine" issues. I worked for a research company providing software engineering support to scientists for a while, and this was a really common problem, and we spent a lot of time trying to come up with fixes and never really found anything ideal.
Case 2 works better, but usually only if the person initially seeking up the project has enough experience with Python to understand what's going on. I've seen projects where Python was added as a quick-n-easy addition to an existing codebase, where the packaging was just a bunch of scripts and makefile commands wrapping virtualenv. This invariably caused problems, but was again difficult to fix later because of subtle behaviours in the hand-written build system.
Case 3, ironically, feels like the easiest to solve here - put everything in a venv and as long as you never need to update anything and never need to share your code with someone else, you're golden.
Compare and contrast this with Cargo or NPM: these tools ensure that you do the right thing from the start, without having to know what additional tools to install, and without needing to think particularly hard about how to set this stuff up. I worked on a project with a Python component and a Javascript component, both set up by the same developer who had minimal experience in either ecosystem, and the difference was like night and day. On the Javascript side we had a consistent, reproducible set of packages that we could rely on well, and on the Python side of things we had pretty consistent issues with dependencies the entire time I worked there.
I’ve been using pixi for a couple months at my work and really loving it. We were already very invested in the Conda ecosystem and it’s a big step up from conda/mamba imo.
Wheels are fine. If everything is a wheel, things just work. It's when you have to deal with source distributions that things get horribly messy because of their fundamentally dynamic nature.
Clickbaity title, esp. when I think the article establishes that uv/pixi are... good? Using those, is it really so bad compared to other commonly used languages?
(Also, I liked learning about pixi; I should learn about the Conda ecosystem at some point. Although at this point, uv has solved my last remaining niggles with Poetry. I understand the more science-y Python projects might not be solvable by uv, but mine are.)
As someone with a non-dev team, getting people to configure their shell to disable the bad defaults in uv (like downloading Python interpreters from the Internet), and using uv consistently instead of pip, yeah, it's pretty bad.
I was very deliberate with "compared to other commonly used languages"; in the end, there's rbenv, nvm, rustup, ... For some time, we've been living in a time when questions like "what is a Linux distribution" are undergoing changes. Frankly, I have no idea how things will play out in the next years, but in the meantime in most of the scenarios I deal with currently, I cannot force Linux distribution uniformity and stuff like containers is undesirable, so rustup/uv are what's most productive to me. Of course, in many scenarios I don't deal with, it's not a good idea.
I do agree getting people to use these tools consistently is hard. In my opinion, most people feel these things are an obstacle, and want to spend as little time as possible on it. In a way, they are right! We are not paid to have smooth environments. Unfortunately, I think we people are bad at estimating how much time we spend due to non-smooth environments.
having used python and “anything else”, yes. in part this is due to the tasks one performs with python - a java project can just happily download pure-java dependencies from maven because it doesn’t need CUDA. still, the end-user experience is the disproportionately bad relative to the language’s mainstream nature.
Well, I think Java is the outlier here. It's awesome that pure-Java dependencies are the norm. Also Java is pretty good with compatibility between versions. And Maven, as controversial as it is, has many accidental benefits. Java has a terrible reputation, but it's not one of the most widespread languages gratuitously.
But still, since around the time that Poetry (and others) appeared, I feel at least there's a couple of decent/acceptable options. Still, you will find horribly broken environments out there, and it's a headache explaining all that is needed to use one of those decent/acceptable options,but there aren't that many better platforms. (Yes, Cargo is awesome. It also was born much later than Python and it learned a lot of lessons from what came before.)
The author of this article seems to put a strong emphasis on (in his opinion) python dependency manager shouldn't be written in python. This in my opinion is a bit misguided, if a python developer wants to fix a bug or contribute a feature if its written in python they can but if its written in another language like uv is written in Rust you now have to learn Rust. Quite honestly what language it's written in shouldn't even be a criteria, its about the tool doing its job.
His comments about nix shows he really didn't use it much before forming an opinion. Which is fine.
All in all, it's a sales pitch to use uv as opposed to balanced review of all the options, but I do agree that python dependency management can be a lot better and it's quirky.
I suppose it's a matter of opinion as to whether the reviews are "balanced", but the author did include a lot of information about the existing tools. He's right, too:
uvis significantly better on all the objective metrics.Having this kind of tooling in Rust is a huge improvement. Not just because it's faster and simpler, but because it avoids the bootstrapping issues: see also https://www.bitecode.dev/p/whats-up-python-uv-disrupts-packaging
I've used Nix and formed an opinion about it: it's just not an appropriate solution for the Python distribution scenarios I've worked with. I'm a little surprised it's even included in the post.
I generally have to agree. If Python had a
uvcompetitor written in PyPy's RPython and translated to a self-contained native binary, then I could see that... but dependency-managing a scripting language in itself is inherently problematic.This is definitely a topic that inspires a lot of strong opinions, but I have trouble reading the same conclusion as the author. I suspect part of this is there are different audiences for Python and they have wildly different expectations for dependencies.
You use Python for ML/AI: You have a million packages, Python is basically the "glue" that sticks your packages together and you should probably use
ulor frankly bite the bullet and just switch to only using containers for local and remote work.You use Python for web stuff. You have "some" packages but this list doesn't change a lot and the dependencies don't have an extremely complex relationship with each other. When you need to do something you should probably attempt to do it with the standard library and only if absolutely necessary go get another package.
You use Python for system administration stuff. Your life is pretty much pure pain. You don't understand virtual environments, you have trouble understanding
pipvs using a Linux package manager to go get your dependency, you don't have a clean environment to work from ever. You should probably switch to a different language.As someone who uses Python primarily for 2, I'm just not seeing anywhere near as much pain as people are talking about.
venvandpip-toolssolves most of my problems and for work stuff I just use containers because that's how the end product is going to be consumed anyway so why bother with a middle-state.In my experience this actually works pretty well, as long as you never touch a Python package manager. Important Python libraries do tend to be available as distribution packages, a README with "apt install py3-foo py3-bar" isn't complicated, and you can make your Python into a proper distribution package if you want automatic dependency management. "System administration tasks" tend not to require a gazillion libraries, nor tracking the very latest version of your libraries.
Mixing distribution packages with pip, poetry, uv, ... is every bit as painful as you describe, though - I agree that one should avoid that if at all possible!
So what you are describing is how people did it for a long time. You write a python script, you install the Debian package for that script system-wide and you bypass the entire
pipecosystem.My understanding from Debian maintainers and from Python folks is this is canonically not the right way to do it anymore. If your thing has Python dependencies you should make a virtual environment, install those dependencies inside of that environment. Like if you want to ship Python as a .deb I think you need to end up doing something like this:
my_python_package/├── DEBIAN/│ ├── control├── usr/│ └── local/│ └── my_python_package/│ ├── venv/ (virtual environment)│ ├── script.py│ ├── other_files_or_scriptsTo be honest, that's one of the big reasons I use Rust for "shell scripting" where I used to use Python. Deployment and maintenance across distro release upgrades are just so much easier with a static binary than with a venv.
I can only imagine that's also why so many people are now using Go for that sort of thing.
(To be honest, I'm hoping that going from Kubuntu 22.04 LTS to 24.04 LTS on this machine will be much less irritating than previous version bumps because of how much I've moved from Python to Rust.)
Yeah, used Python professionally for a decade. Never had a single problem. Came to think that people railing against Python dependency management had a "them problem".
Python packaging has its issues but have you seen Node where there are a handful of wildly incompatible ways to build/include libraries and nobody can push anything forward?
I have been finding success with
uv runfor these users and either uploading packages to the registry for them to execute withuvxor inline script metadata for dependencies.I love being able to tell somebody to just go install pipx and then
pipx installa tool directly from a git repo. It's so straightforward! I'm very glad thatuvsupports this too. It's quickly becoming the only python dependency tool I need.I use python for 2.
It is a nightmare. Venv make no sense and I have yet to understand how to use pip-tools. That is after having written and fixed package managers in othet languages and even being a nixpkgs maintainer.
Even nix is easier to write and use that venv. As long as venv are part of the UX of your solution, this is not going to fly.
Can you clarify how venvs make no sense to you? They seem to be the one part of Python that actually make sense to me, and bring some sanity. (I still prefer Nix, too.)
I'm surprised? This isn't me attacking you, just genuine curiosity. My primary issue with onboarding interns or junior folks onto python projects has been explaining that
venvexists, but once they grasp the basics of it seems to "just work". Especially withpip-compileandpyproject.tomlthere doesn't seem to be a ton of places for it to go wrong, but I'd love to know what about it doesn't work for you.In my experience, case 1 usually results in "it works on my machine" issues. I worked for a research company providing software engineering support to scientists for a while, and this was a really common problem, and we spent a lot of time trying to come up with fixes and never really found anything ideal.
Case 2 works better, but usually only if the person initially seeking up the project has enough experience with Python to understand what's going on. I've seen projects where Python was added as a quick-n-easy addition to an existing codebase, where the packaging was just a bunch of scripts and makefile commands wrapping virtualenv. This invariably caused problems, but was again difficult to fix later because of subtle behaviours in the hand-written build system.
Case 3, ironically, feels like the easiest to solve here - put everything in a venv and as long as you never need to update anything and never need to share your code with someone else, you're golden.
Compare and contrast this with Cargo or NPM: these tools ensure that you do the right thing from the start, without having to know what additional tools to install, and without needing to think particularly hard about how to set this stuff up. I worked on a project with a Python component and a Javascript component, both set up by the same developer who had minimal experience in either ecosystem, and the difference was like night and day. On the Javascript side we had a consistent, reproducible set of packages that we could rely on well, and on the Python side of things we had pretty consistent issues with dependencies the entire time I worked there.
I’ve been using pixi for a couple months at my work and really loving it. We were already very invested in the Conda ecosystem and it’s a big step up from conda/mamba imo.
Wheels are fine. If everything is a wheel, things just work. It's when you have to deal with source distributions that things get horribly messy because of their fundamentally dynamic nature.