On BSD systems, the three paths are clearly distinct:
/ is for things that are needed in single-user mode. If you are booting from a RAM disk, or a small disk and network mounting everything else, things needed for boot go here.
/usr/ is for other binaries from the base system. Things here are all part of the core install. Unless you’re building an appliance, the things that you can expect to find here should be consistent.
/usr/local is for things that are installed from ports / packages. These are all managed by tools, but they will vary between systems. If you depend on anything here, you need to document it as something that needs to be installed.
I tend to install things I’ve compiled myself in /opt/{thing name and version}. This makes uninstalling a thing easy (use rm -rf) and makes it easy to have two or more versions of the thing installed. I have a little loop in my profile that adds all of the bin / sbin directories under here to my path. Things that depend on libraries installed here are linker with rpath directives. I believe Homebrew uses a similar structure and I wish more package managers did.
I really liked the way PC-BSD installed things. Every app (PBI was for user-facing things, not for their dependencies) installed in a separate directory containing all of its dependencies butit used hard links for all libraries that it depended on, so you only had one copy. Everything was linked using rpath and $ORIGIN.
The thing I didn’t like was that it downloaded the complete thing, rather than the packages that it used. Every KDE app downloaded a complete copy of the KDE libraries, which was then discarded and replaced with a hard link to an existing one. This would not be great now, but was much worse when home Internet connections were typically <10 Mb/s.
Things like Flatpack at least have a base image with the KDE libs installed.
From a packaging perspective, I have no trouble overwriting users files if they have put it in /usr not /usr/local
As a user, I am terrified of a packager choosing the same name as I did and overwriting my scripts.
As somebody who has systems that other people use, I would be very wary of putting anything in a system-controlled location if my users are the ones writing it.
It’s been years and years now, yet I still can’t understand how we ended up with /usr/bin as the actual path for binaries, with /bin symlinked to it…
On a side note, I wish overlayfs could be used by non-provilegzd users as a poor man’s mount namespace ala plan9. $PATH could then be ~/bin, and I’ll decide what to put in there by simply mounting layers on top of it.
Disk space in the 70s IIRC: originally all binaries were in /bin, but as the collection grew machines started running out of drive space.
/usr was on a separate larger drive (originally for user directories à la /home or /Users nowadays) and was much larger, so binaries started being overflowed over there, with /bin reduced to boot requirements (whatever was needed to mount /usr or do very basic single-user administration to repair the machine if /usr could not be mounted).
Yeah that’s the explanation for the existence of /usr/bin (for anyone interested, it was depicted on this thread from the BusyBox mail list). But my questionning is more about why /usr/bin was chosen as “the one true path” for binaries by distro packager.
But all of these arguments essentially boil down to “now /usr is mandatory anyway and we already put stuff in there so we might as well only keep this one”.
The only argument that is somewhat okay-ish is that you could then mount /usr read-only for “security reasons”. But nobody ever does that because it is a real pain to manage (eg, installing new packages require a remount in RW). IMO they should have used /bin and /lib for the merge and symlinked their /usr counterpart to it. I guess it was mostly a trend and distro packagers did not want to bother that much…
The motivation for the Fedora / Systemd crowd was to enable ChromeOs style immutable root filesystems with A/B partitions. They did make use of it with CoreOS and Silverblue. From what I understand SteamOS is taking advantage of this too.
you could then mount /usr read-only for “security reasons”. But nobody ever does that because it is a real pain to manage (eg, installing new packages require a remount in RW).
Shouldn’t the package manager be able to do the remounting as necessary? The Nix package manager does approximately that, for example.
There are other arguments: you could share /usr with other systems, you could wipe-out anything outside /usr (and /boot) and still get a functional system. For me, it makes sense that everything system-related is in /usr, despite the historical naming.
Try to remove the /bin or /lib symlink and tell me how functional your system is then 😉
There’s a reason all distro have been keeping these “backward compatible” hacks for 12 years.
I could somehow buy the “share” argument, however I’ve yet to meet an environment doing that in real-life. Disk space is cheap and multiple systems simply get their own /usr, even when they’re identical. And even then, you could just mount it to /usr/remote_host and add that to your path.
I couldn’t live with ~/bin{,-go,-js,-py,-rust} and ~/.golang polluting my $HOME. At least nest them!
Also ~/.local/bin is definitely not Python specific.
Generally, I do the opposite of creating symlinks: setup env vars/config like $GOBIN to change what the tools do.
To each their own, but I don’t think “modern” really makes sense outside of deduping symlinks from /.
I agree, I should have nested the path and I don’t like tools polluting my $HOME either.
Problem is, there is no easy way to configure all package manager to install to a specific directory, with {go, cargo, npm, pip, …} install xxx, each one will install at a different place.
I propose a better solution, systemd-pathd.
It’s simple: systemd will offload all this logic and keep track of both user and system binary paths. All we need in .profile is:
tbh it’s not that bad, it usually just adds your nix profiles, which are symlinked directories much like /usr/bin (and you usually have only one or two at the same time). And there’s no /usr/bin on NixOS (unless you directly add store paths to $PATH, then that’s nightmare fuel).
To go further, I remove /usr/games since I run GUI programs (like games) through their .desktop entries. These entries are located at ${XDG_DATA_DIRS}/applications.
Games that end up in /usr/games such as those packaged in bsdgames (file listing) are frequently not GUI programs.
Not that I am especially opposed to you removing that from your path regardless.
I don’t understand why distros don’t put ~/.local/bin in their PATH env and integrate it into build tools and other linstall tools. It’s the fd.o prescribed location for user binaries and fits in well with the rest of the freedesktop standard locations. Adopting this stuff has helped me keep my homedir organized a bunch.
It’s not great for security to have a directory in your PATH that is writeable by you. It allows a directory traversal vulnerability to be turned into a full compromise. An attacker just needs to drop a binary called ls or similar in you well-known binary directory and they have arbitrary code execution on your machine with your privilege level (which, even if it isn’t root, is enough to read and modify all of your files).
Most of the vulnerabilities that let you write to the wrong place let you write an entire file. Replacing .profile would likely be noticed immediately because it would change prompts, aliases, and so on. Replacing ls with something that installed a malicious daemon, erased itself, and exec’s the real ls would not.
but how many people actually use .profile and not .bash_profile or their shells equivalent?
you could also probably replace the completion script for cat or another common-but-simple command and achieve a similar effect without being overly noticable.
but how many people actually use .profile and not .bash_profile or their shells equivalent?
I don’t understand your point. Drop a replacement .bash_profile in and people will notice.
you could also probably replace the completion script for cat or another common-but-simple command and achieve a similar effect without being overly noticable.
I’m not sure how your profile is configured, but I’ve never seen a system that sourced completion scripts from anywhere user-writeable by default.
,> With this setup, I can more easily verify if the correct binary is called.
It might be shorter, but you also made it more dynamic.
PATH usually survives for many years and stuff is just appended unless making a dedicated clean up.
I feel like while it might be shorter on overage adding conditions and thereby making it dynamic could be complicated.
Granted these are very simple conditions so it might work.
I don’t really look at PATH much and if I do it’s fine to open my config and see what it is made up of i in a more elaborate way. There I also have comments I wrote.
Maybe I just got lucky to never have to worry in the past 20ish years.
I got anxiety just reading this.
On BSD systems, the three paths are clearly distinct:
I tend to install things I’ve compiled myself in /opt/{thing name and version}. This makes uninstalling a thing easy (use rm -rf) and makes it easy to have two or more versions of the thing installed. I have a little loop in my profile that adds all of the bin / sbin directories under here to my path. Things that depend on libraries installed here are linker with rpath directives. I believe Homebrew uses a similar structure and I wish more package managers did.
Back when I used a Mac, MacPorts also installed packages to
/opt, and the Nix package manager uses/nix.I really liked the way PC-BSD installed things. Every app (PBI was for user-facing things, not for their dependencies) installed in a separate directory containing all of its dependencies butit used hard links for all libraries that it depended on, so you only had one copy. Everything was linked using rpath and $ORIGIN.
The thing I didn’t like was that it downloaded the complete thing, rather than the packages that it used. Every KDE app downloaded a complete copy of the KDE libraries, which was then discarded and replaced with a hard link to an existing one. This would not be great now, but was much worse when home Internet connections were typically <10 Mb/s.
Things like Flatpack at least have a base image with the KDE libs installed.
From a packaging perspective, I have no trouble overwriting users files if they have put it in /usr not /usr/local
As a user, I am terrified of a packager choosing the same name as I did and overwriting my scripts.
As somebody who has systems that other people use, I would be very wary of putting anything in a system-controlled location if my users are the ones writing it.
It’s been years and years now, yet I still can’t understand how we ended up with
/usr/binas the actual path for binaries, with/binsymlinked to it…On a side note, I wish
overlayfscould be used by non-provilegzd users as a poor man’s mount namespace ala plan9.$PATHcould then be~/bin, and I’ll decide what to put in there by simply mounting layers on top of it.Disk space in the 70s IIRC: originally all binaries were in /bin, but as the collection grew machines started running out of drive space.
/usr was on a separate larger drive (originally for user directories à la /home or /Users nowadays) and was much larger, so binaries started being overflowed over there, with /bin reduced to boot requirements (whatever was needed to mount /usr or do very basic single-user administration to repair the machine if /usr could not be mounted).
Yeah that’s the explanation for the existence of
/usr/bin(for anyone interested, it was depicted on this thread from the BusyBox mail list). But my questionning is more about why/usr/binwas chosen as “the one true path” for binaries by distro packager.Back in 2012 I was running Arch Linux when this merge happened, and the reasoning behind it never convinced me: merge /bin, /sbin, /lib into /usr/bin and /usr/lib. Other distros made similar comments, eg. Fedora: The Case for the /usr Merge.
But all of these arguments essentially boil down to “now /usr is mandatory anyway and we already put stuff in there so we might as well only keep this one”. The only argument that is somewhat okay-ish is that you could then mount
/usrread-only for “security reasons”. But nobody ever does that because it is a real pain to manage (eg, installing new packages require a remount in RW). IMO they should have used/binand/libfor the merge and symlinked their/usrcounterpart to it. I guess it was mostly a trend and distro packagers did not want to bother that much…The motivation for the Fedora / Systemd crowd was to enable ChromeOs style immutable root filesystems with A/B partitions. They did make use of it with CoreOS and Silverblue. From what I understand SteamOS is taking advantage of this too.
Shouldn’t the package manager be able to do the remounting as necessary? The Nix package manager does approximately that, for example.
There are other arguments: you could share /usr with other systems, you could wipe-out anything outside /usr (and /boot) and still get a functional system. For me, it makes sense that everything system-related is in /usr, despite the historical naming.
Try to remove the
/binor/libsymlink and tell me how functional your system is then 😉 There’s a reason all distro have been keeping these “backward compatible” hacks for 12 years.I could somehow buy the “share” argument, however I’ve yet to meet an environment doing that in real-life. Disk space is cheap and multiple systems simply get their own
/usr, even when they’re identical. And even then, you could just mount it to/usr/remote_hostand add that to your path.Also so that in a computer pool etc /usr/bin with shared applications for everyone could easily be a network drive.
I couldn’t live with
~/bin{,-go,-js,-py,-rust}and~/.golangpolluting my$HOME. At least nest them!Also
~/.local/binis definitely not Python specific.Generally, I do the opposite of creating symlinks: setup env vars/config like
$GOBINto change what the tools do.To each their own, but I don’t think “modern” really makes sense outside of deduping symlinks from
/.I agree, I should have nested the path and I don’t like tools polluting my $HOME either. Problem is, there is no easy way to configure all package manager to install to a specific directory, with {go, cargo, npm, pip, …} install xxx, each one will install at a different place.
Yeah it’s definitely a whack a mole situation!
Hopefully with time it gets better, but XDG dirs are not the best respected standard at the moment.
you certainly could do this, I’m not really clear why you’d want to though?
There’s also the
whichcommand, which I think would be easier than doing all this work.I propose a better solution, systemd-pathd.
It’s simple: systemd will offload all this logic and keep track of both user and system binary paths. All we need in .profile is:
(/s)
Welcome to Nix? (/s??)
tbh it’s not that bad, it usually just adds your nix profiles, which are symlinked directories much like
/usr/bin(and you usually have only one or two at the same time). And there’s no/usr/binon NixOS (unless you directly add store paths to$PATH, then that’s nightmare fuel).Yeah, I guess that was my point! :)
Games that end up in
/usr/gamessuch as those packaged in bsdgames (file listing) are frequently not GUI programs.Not that I am especially opposed to you removing that from your path regardless.
Meanwhile, on Windows:
With Zsh, I am using this approach:
https://github.com/vincentbernat/zshrc/blob/master/zshenv#L17
I don’t understand why distros don’t put ~/.local/bin in their PATH env and integrate it into build tools and other linstall tools. It’s the fd.o prescribed location for user binaries and fits in well with the rest of the freedesktop standard locations. Adopting this stuff has helped me keep my homedir organized a bunch.
It’s not great for security to have a directory in your PATH that is writeable by you. It allows a directory traversal vulnerability to be turned into a full compromise. An attacker just needs to drop a binary called ls or similar in you well-known binary directory and they have arbitrary code execution on your machine with your privilege level (which, even if it isn’t root, is enough to read and modify all of your files).
I believe this is why a lot of platforms default to system paths first?
although, if you can write to the filesystem, you can also just write to .profile, so it’s a bit of a moot point.
They do?! That would be annoying!
~/binneeds to be first in$PATHso you can override system binaries.Most of the vulnerabilities that let you write to the wrong place let you write an entire file. Replacing .profile would likely be noticed immediately because it would change prompts, aliases, and so on. Replacing ls with something that installed a malicious daemon, erased itself, and exec’s the real ls would not.
but how many people actually use .profile and not .bash_profile or their shells equivalent?
you could also probably replace the completion script for
cator another common-but-simple command and achieve a similar effect without being overly noticable.I don’t understand your point. Drop a replacement .bash_profile in and people will notice.
I’m not sure how your profile is configured, but I’ve never seen a system that sourced completion scripts from anywhere user-writeable by default.
Fish, at least, will automatically load completions from
~/.config/fish/completionsby default.my point is that you can overwrite
.profilewithout many people noticing, since they store all their actual configuration in.bash_profileinstead.and for many attacks, it doesn’t matter if you immediately notice something is wrong, by the time the malicious code is run, it’s already game over.
,> With this setup, I can more easily verify if the correct binary is called.
It might be shorter, but you also made it more dynamic.
PATH usually survives for many years and stuff is just appended unless making a dedicated clean up.
I feel like while it might be shorter on overage adding conditions and thereby making it dynamic could be complicated.
Granted these are very simple conditions so it might work.
I don’t really look at PATH much and if I do it’s fine to open my config and see what it is made up of i in a more elaborate way. There I also have comments I wrote.
Maybe I just got lucky to never have to worry in the past 20ish years.