I have grievances against OpenBSD file system. Every time OpenBSD crash, and it happens very often for me when using it as a desktop, it ends with file corrupted or lost files. This is just not something I can accept.
!!!
For comparison, ext3, with journaling, was merged into Linux mainline in 2001.
I developed embedded devices whose bootloader read and wrote ext4 files, including symlinks. There is really no excuse not to have a journaling file system on a system that’s larger than a fingernail.
For OpenBSD it may be as similar reason as why they got rid of Bluetooth. Nobody was maintaining that code. It got old, stale. So they got rid of it. I generally actually like this approach, sadly you lose functionality, but it keeps the entire codebase “clean” and maintained.
My guess is that they need people to work on the FS issue. Or, rather they don’t have anyone who cares enough about it onboard to actually write the code in a way that is conductive to OpenBSD’s “style”. Could be wrong, but that’s my assumption.
The pragmatic thing to do would be to grab WAPBL from NetBSD since NetBSD and OpenBSD are still relative kin. Kirk McKusick still maintains UFS on FreeBSD so the SU+J works well there but it would be a lot of work to pull up OpenBSD’s UFS.
IIRC WAPBL still has some issues and is not enabled by default.
Its primary purpose is not to make the filesystem more robust but rather to offer faster performance.
I’ve never experimented with SU+J, but I’d like to hear more feedback on it (:
I’m not sure how deep WABPL goes but a journal helps you to close some corruption events on an otherwise non-atomic FS by being more strict with the sync and flush events and without killing performance. You can also journal data which is an advantage of taking this approach, although I don’t know that WAPBL offers this currently. Empirically NetBSD UFS+WAPBL seems fairly reliable in my use.
SU+J orders operations in an ingenious way to avoid the same issues and make metadata atomic at the expense of code complexity, the J is just for pending unlinks which otherwise have to be garbage collected by fsck. A large, well known video streamer uses SU+J so it is well supported. Empirically SU+J is a little slower than other journaling filesystems but this might be as much implementation and not algorithm.
Nice, I had not seen that thread. I think the data journaling they are discussing would be important for OpenBSD as for instance on FreeBSD UFS is used in specific scenarios like embedded devices or fail-in-place content servers and ZFS is used anywhere data integrity is paramount. WAPBL was created in response to the complexity of SU+J as it seems OpenBSD was also impacted by. For that and easier code sharing I would be inclined to go the WAPBL direction, but there may be other merits to the SU+J direction in terms of syncing FFS and UFS against FreeBSD.
This is really surprising. I thought this was one of those instances of OP “handling things wrong”, but actually it doesn’t seem OpenBSD natively supports anything else than FFS and FFS2 (and without soft updates, like above).
I use OpenBSD on both my personal X220 laptop and my ThinkCentre Tiny desktop. Love the experience on both of those machines. I do see the pain points brought up by the author though (lack of bluetooth support specifically). Luckily, those aren’t deal breakers for me - but I completely understand.
I found myself gravitating to Debian unstable on most of my personal laptops. It offers a reasonable tradeoff of “industry standard” and not being frustratingly out of date without using a boutique OS (I’ve used Gentoo and Arch in the past which can be fun but also consume a lot of time) or quasi-commercial OS (Ubuntu, Fedora, OpenSUSE) with the regular rug pulls and weird forced decisions. I’ve never really been drawn to the different takes on state management or partitioned security OSes and Debian lets me focus on getting whatever else done.
I use FreeBSD on a variety of systems and it doesn’t suffer from these same drawbacks, especially on servers, but it has others.. no current wifi standards support, suspend/resume is very hardware dependent, and graphics are always a sudden unexpected learning event. If you have a desktop where standby doesn’t matter, the official nvidia driver is very stable and performant and I typically use Ethernet - I do my best work on this system.
I have some NetBSD systems (in particular a T480) and really like it but it needs more developer help to offer a compelling mobile option. I do run a couple OpenBSD as well but wouldn’t want to use it for a desktop for performance and filesystem reasons.
I’m pretty wrapped up in FreeBSD so there will be some implicit biases for a variety of reasons.
If I use a five dollar word, I would say FreeBSD has much better orthogonality than any Linux distro (although some approach it). Enough of this is cultural that it isn’t an accident, but to borrow a term src is a “monorepo” (the base system is all in one) and this causes some natural alignments between the build system, compiler, libc, administration utilities, man pages and kernel. There seems to always be some low temperature debate on what should be in there, for instance maybe it doesn’t make sense to have the compiler, but having the kernel, libc, man and base utilities in one is a big benefit versus Linux. It makes evolving interfaces a lot more natural, and yet FreeBSD still provides excellent backwards compatibility to at least 4.x.
What this means practically is that if you used a FreeBSD system 20 years ago and today there would not be a lot of surprises, and mostly pleasant ones like a better filesystem (ZFS) and block abstraction (geom) now being the norm.
For system development, it is really low friction versus others. Especially the Linux kernel which intentionally moves and breaks interfaces all the time to make out of tree development painful.
I benefited greatly from getting involved in FreeBSD, even when I was working strictly on Linux systems, so I would recommend it to anyone that is interested in operating systems. It will channel you in different directions than the loudest norms today like Docker and Kubernetes, although there is no technical reason these can be implemented directly or analogously.. and this isn’t always a bad thing. Keeping an open mind and noting what is done well or what is lacking will help in both directions when comparing or deciding when to use what.
In terms of security, do BSD Jails offer a better solution to isolate applications?
Linux is a bit behind in this respect, and the X Window System makes the problem worse by essentially letting any graphical application to see key events from other applications or inject spurious ones.
Linux has so much, it is hard to pin down any fair comparison. I view jails as more of an administrative concern, like Linux namespace+cgroup combo, and not a hard security feature although these things don’t hurt to help separate concerns. Landlock is a compelling option on Linux and is inspired by FreeBSD’s capsicum (for some reason, when Google sponsored a direct port, that didn’t get in) – for something like a server process or even a browser I think these are good enough for the masses.
If you really care about isolation the Qubes approach is a bit more plausible because a virtual machine, even though an immense abstraction, has a much easier to reason about boundary in the hardware than any pure kernel approach. The CHERI project, which is FreeBSD adjacent, aims to amend this extreme by moving capabilities into hardware - which might make you trust a kernel construct like jails more.
Linux is a bit behind in this respect, and the X Window System makes the problem worse by essentially letting any graphical application to see key events from other applications or inject spurious ones.
X has all the capabilities to block this (this is the difference between ssh -X and ssh -Y; -X uses an “untrusted” connection which puts it in an isolation group, -Y doesn’t, and there’s more beyond that which is even harder to use like the access control extension, or outright running nested servers for even more), it just isn’t turned on typically. Some distros even configure it out entirely in their builds. But if you really wanted to, there are options available.
That is probably the strongest solution - that article mentions a couple of the other options but states the nested servers is the tightest barrier, which I’d agree with. Though interesting to note that it then punches holes in that barrier anyway in the name of convenient usability, like the clipboard. That’s the problem with these isolation schemes - you usually do end up poking a lot of holes in it anyway since a fully isolated thing is a pain to use.
I’m honestly surprised you said Debian unstable. Do you run into many issues with broken packages or broken upgrades?
I’ve used Testing for about 15 years, but the few times I’ve tried unstable ended up with broken stuff at the worst possible times. The only problems I have with testing involve proprietary firmware (thanks Apple and Nvidia…) either moving around or dropping support for things.
I have not experienced an unusable system in 2 years. The case for unstable is that testing does not receive any security priority so you can be running vulnerable versions longer. Investing some time into setting up Btrfs snapshots would probably be wise but I haven’t done so myself yet.
I check apt dist-upgrade for anything peculiar before accepting, in particular there are sometimes days or even weeks where a dependency chain may be broken. An ’apt upgrade` instead will usually work around that until it is fixed. This cycle towards 13 has had some big events, like 64-bit time, rearranging kernel firmware packages, and Qt changes.. Plasma 6 might be a big one too. I would expect future cycles to be a bit more tame t64 was a big one.
On Qubes OS: I used it for several years, and really liked it. Though it was always a little bit of a performance slog, especially because I abuse my laptop’s resources a lot (mostly with way, way too many open browser tabs, but also with too many qubes) and hit a lot of bugs because of this. But the real problem was that it was bad for my mental health: I found it turned my professional paranoia up to a pretty extreme level that was mostly worth the tradeoff, but clearly a bit detrimental. Simple operations like copying and pasting all of a sudden became fraught, because I’d ask myself questions like “is it really worth it to copy and paste this text from a less-secure qube into a more-secure qube, or is it short enough that I can just type it out again and avoid an escalation vector?”
I switched to Fedora Silverblue “just to try it out” (I really did intend to only try it for a little bit of time) and haven’t looked back since.
I enjoy reading Solene’s articles, because she always takes a pragmatic, but with an eye on security, approach to things.
I especially enjoyed reading Solene’s articles on OpenBSD, as she was amongst the very few writers out there that discuss about OpenBSD from a non-server point of view.
Thus, I am very sad to read about her leaving the OpenBSD ecosystem, however, reading her article I couldn’t agree more with most of her observations, especially those that also impact OpenBSD deployments as a service platform:
lack of a modern file-system – and I don’t even want fancy things like compression or encryption (at the FS level), but at least some modicum of performance and checksums;
lack of lock-free network stack – even though OpenBSD is favored for network appliances, only recently they’ve started working on lock-free (i.e. performant) network stack; I’ve done once a benchmark between Linux/FreeBSD/OpenBSD and although I don’t remember right now the figures, the performance penalty was quite significant;
(this one isn’t their fault) lack of (first-class) support for newer programming languages, especially Rust;
and, as Solene mentions, there are many small paper-cuts, but when combined sways me against OpenBSD;
Granted, the OpenBSD has done wonders with its small and underfunded team. Unfortunately, “the world” wants “shiny” things…
only recently they’ve started working on lock-free (i.e. performant) network stack
Don’t conflate these two. Fine-grained locking can outperform lock-free data structures in the network stack on low core counts. FreeBSD started moving to lock-free data structures a while ago after doing a lot of profiling here. If you have under 8 cores, fine-grained locking is often faster. For 8-16, it’s often not clear which is a winner. Above 16, lock-free can be a win and becomes bigger as core counts scale (data based on decade-old hardware, might be different now).
Unless you’re using in the 10Gb/s range or above or have an lot of concurrent connections, and you’re using a lot of cores for networking, you probably won’t see a big speedup from moving from fine-grained locking to lock-free approaches.
That said, OpenBSD was quite late moving away from a single giant lock. This just scalability problems around 4 cores. I remember a presentation from some OpenBSD folks at a conference 8-10 years ago showing that OpenBSD’s pf was faster than FreeBSD’s. This irritated a bunch of FreeBSD network folks because the presenter didn’t mention that it was on a single-core Soekris system. Adding more cores gave an almost linear speedup in the FreeBSD version (at least up to 8 cores), whereas it gave a slight slowdown on OpenBSD.
I agree that for most “normal” scenarios, the network related performance overhead shouldn’t count much, because the majority of the time would be spent in doing template rendering or waiting for the DB.
But, for my particular case which was network intensive, the overhead would have been felt. Especially since my initial scalability plan resorts much on vertical scalability (i.e. get a bigger VM / box).
I think I wasn’t clear. I am trying to say that ‘fast’ and ‘lock-free’ are not the same thing. If you have a single core spending close to close to 100% of its time in the network stack, coarse-grained (a.k.a. Giant) locking is probably fine. If you have 2-4 cores in this state, fine-grained locking will probably be the fastest approach, assuming that the locks are in sensible places.
Up to about 16 cores, it’s hard to predict and over 16 cores you’ll probably see a speedup from lock-free data structures, thought this isn’t guaranteed (and with a SmartNIC that supports multiple queues, you may see better performance with simple per-core flow routing as some userspace network stacks do, with no cross-core communication at all in the network stack other than for marking in-use ports).
Way back when WhatsApp first announced that they were serving over a million active connections from a single FreeBSD server, the network stack was using fine-grained locking. When Netflix first announced that they were saturating dual 10 GigE links with TLS on a single FreeBSD machine, the network stack was using some lock-free data structures (and some fine-grained locking) but the big speedups came from the combination of KTLS and sendfile / aio_sendfile.
The OpenBSD network stack may not scale in the direction that you need, but that isn’t necessarily to do with the lack of lock-free data structures.
only recently they’ve started working on lock-free
This isn’t recent, it’s just been going very slowly over many years because only one or two people are working on it.
lack of (first-class) support for newer programming languages, especially Rust*
What is lacking in OpenBSD’s Rust support? We have up-to-date packages for Rust, Go, Crystal, all the Ruby versions, Zig, Hare, and probably some other trendy new languages that I don’t recognize in a quick git log of /usr/ports/lang. Usually if we have a package, it works properly just the same as on other platforms.
Indeed, OpenBSD does technically support Rust, as in having a (ports) package for it. (And it seems today it’s even up-to-date, since two years ago when I checked it was a few versions behind.)
However, at least in the Rust ecosystem, having “support” implies having at least rustup working for a given platform. Why?
not only does one get a pre-built toolchain (so one doesn’t have to build it from ports), but one can choose which version of Rust to use; (even Rust has, from time-to-time regressions, thus the need for an older compiler;) (or for example one could use one of the nightly builds for certain features;)
moreover, once a target has rust-up support, a developer on another platform can, if not build and link, then at least check if the application would compile for OpenBSD;
Here are a few reasons, cited from the rustup project, on why it doesn’t yet support OpenBSD: https://github.com/rust-lang/rustup/issues/2168#issuecomment-678608631 – mainly lack of binary bootstrapping of OpenBSD binaries from Linux, and lack of OpenBSD compatibility between versions;
Furthermore, when I tried to build my Rust project on OpenBSD (after installing the toolchain from the ports), although I didn’t have too many dependencies, it did fail to build due to some C-based dependency. And, I’m certain if I would have made that work, I would have stumbled upon the next one, etc. Why? Because very few packages are actually tested against anything else than Linux / OSX and perhaps Windows.
Now, I’m not criticizing the OpenBSD developers for this, because in the end it’s the Rust project’s (or community’s) job to make sure their language works on a certain platform they want to support. And the same for various libraries.
However, given this is a recursive / feedback-loop type of problem (i.e. libraries don’t get tested, because language is not supported, because there aren’t much libraries that work, loop), the chain must be broken (or started) somewhere, and I would say that if one at least can cargo check --target x86_64-unknown-openbsd their code against OpenBSD it would be a huge win.
For example, in one of my projects, I can build (as in link an executable) at least for Linux, Android, OSX and Windows (all x86-64 and aarch64), plus WASM32-WASI, and although I can’t yet build a FreeBSD binary, I can at least check against it, so that if one wants to build my tool on FreeBSD, some work would have been already done.
If I want to test my code against OpenBSD, I can’t without also running OpenBSD myself.
Sure, you are right, if I don’t also run OpenBSD, I can’t make sure that my tool actually runs on OpenBSD.
Then, if I also don’t run FreeBSD, I can’t make sure that my tool also runs on FreeBSD.
(Perhaps I should also try NetBSD? Or perhaps my time is best spent for DragonflyBSD?)
Then, if I’m at it, if I don’t run OSX, then I can’t make sure that my tool runs on OSX. Wait, there are two main variants now? ARM and Intel based ones? Well, I’ll now have to buy myself two $1K+ laptops to make sure my open-source project works on OSX.
So, now that I’m a couple $K into it, why not add another $K to buy a machine able to run Windows 11, so I can make sure that my tool works on Windows. (What, Windows also has an ARM variant now?)
Or, the alternative where I save a couple of $K just for the hardware, I just make sure it works on my current Linux distribution, and whomever wants to use it on OpenBSD (and other OSs) can just send back bug-reports if it fails to build or run.
I’m sorry for the sarcasm, but I wanted to convey that OpenBSD isn’t the only OS out there, and one can’t run natively all targeted operating systems without eating a lot of energy (and possibly $) and time, that could have been better spent improving one’s project.
Thus, let’s meet half-way: I cargo check for many platforms (this ensures that the Rust code is portable), and if the target has the cross-build tooling for Linux (which OSX, Windows and Android does), I’ll also build it.
And if I cargo check my tool, which has (as is the Rust way) a few tens of dependencies, if anyone of them also fails I’ll open bug reports on their projects, thus just one tool checking for OpenBSD translates to multiple projects being also checked.
I just make sure it works on my current Linux distribution, and whomever wants to use it on OpenBSD (and other OSs) can just send back bug-reports if it fails to build or run.
Far from unreasonable, really, though I note that it’s still in the sarcasm section of your post.
That statement of mine was more “in between” sections, and although not unreasonable as you say, is what actually happens in open-source nowadays: most developers just test and run their code on their setup
Which actually means exactly one data point from the cartesian product of (kernel(*version?) * distribution(*version?) * processor(*generation?) * c-library(*version?) * compiler(*version?)).
Thus, in reality, a developer can only actually “hope” that his code actually runs on the myriad possible combinations out there.
(this one isn’t their fault) lack of (first-class) support for newer programming languages, especially Rust;
I think it is, at least partially. OpenBSD deliberately refuses to have not only a stable OS ABI (which is fair, most OSes other than Linux don’t) but also a stable API. They reserve and make use of the right to break libc headers, even in minor releases.
This makes it impossible for third-party software like Rust to target OpenBSD generically - you need to port it to every new OpenBSD minor release. And I’m not making this up, here are OpenBSD devs asking Rust people for this exact thing and getting told essentially “sorry, no, you’re not special”: https://github.com/rust-lang/libc/issues/570.
Edit: To clarify, I’m not saying OpenBSD are “ wrong” here, just making the tradeoff clear. They are choosing the simplicity and maintenance ease of not having to deal with backcompat headaches over good support for third-party software.
Does OpenBSD even have minor releases? As far as I can tell they just increase the version number by 0.1 every six months, and the number doesn’t indicate anything about what’s changed other than the date.
Every time OpenBSD crash, and it happens very often for me when using it as a desktop
I am curious about that. I think OpenBSD might have never crashed on me. Windows (also post-XP), macOS, Linux, FreeBSD, NetBSD, DragonFly (though that was hardware related) all did multiple times though.
Is this due development? Is this hardware related?
Just surprised about that particular one. All the others feel like “sure, if that’s what you want then OpenBSD is probably not a good choice”.
I am curious about that. I think OpenBSD might have never crashed on me. Windows (also post-XP), macOS, Linux, FreeBSD, NetBSD, DragonFly (though that was hardware related) all did multiple times though.
Very similar experience. Don’t think I’ve ever had a crash. Her experience might also have something to do with solene being an OpenBSD dev, so testing untested stuff possibly leads to some I stability? That’s an assumption on my part.
I’ve used OpenBSD on my personal laptops (and work when I can) for the last 7 years at least. Installation, configuration and system related operations are a joy.
But the less then ideal docker through vm’s way of doing docker does stop me from using it for many contracts and in some contexts it can be very slow. I almost never have crashes and I dont remember losing any files, though I always have to do fsck when my battery runs out, and that happens pretty often.
If I move on from OpenBSD, the likely suspects are void linux, guixSD or Genode SculptOS.
Wow, I don’t even remember writing that. I just use apmd’s -Z option now to automatically hibernate when the battery is super low, with a script in /etc/apm/hibernate that tries to warn me with Xdialog.
I too use OpenBSD sparingly, but when I can get away with it, I use it.
Most of the time I’m using FreeBSD, MacOS or Linux. I want to get off of MacOS, but I’m not in any particular hurry. I just worry that Apple will eventually ruin enough that it’s no longer useful as my desktop.
I’ve used OpenBSD as my primary desktop since 2001, but I get Solene’s points, and I would like BlueTooth back again (I don’t have the skills to implement it, and I have enough work arounds that it’s a nice to have…), and over the years I’ve seen crashes, but luckily I’ve not lost any data, and usually I’m back working again quickly, even with an fsck of the ffs.
I’ve also never really played computer games, as life is too short as it is, but I have run minecraft servers on OpenBSD and installed many of the games in packages for my kids to play with over the years.
OpenBSD isn’t even really supposed to be a desktop OS. I’d say it’s more like router firmware. I’m always shocked when someone actually implies they do or have been using it as a desktop OS.
And yes, I know there’s going to be someone who insists they also use it. I’ve also seen people try to use Windows XP x64 Edition well into 2014. Trust me, I have seen no shortage of questionable life choices.
The author of this was previously on the OpenBSD development team. OpenBSD devs tend to dogfood their own OS, so of course she would have used it as a desktop.
This isn’t really true. A few porters do huge amounts of work to keep (among other things) KDE and Chromium and Firefox available for OpenBSD users, and not insignificant work goes into making the base system work (more or less) on a decent variety of laptops. It’s less compatible than Linux but for a project with orders of magnitude less in resources than Linux it does pretty good. But I guess we’ve finally reached the Year of the Linux Desktop if we’re now being shocked that someone would have a BSD desktop.
I would say that the vast majority of OpenBSD developers are using it as their primary OS on a desktop or laptop. I am shocked (well not really anymore, but saddened) that developers of other large mature operating systems don’t use it as their primary OS. If you’re not using it every day, how do you find the pain points and make sure it works well for others?
We have reasonably up-to-date packages of the entire Gnome, KDE, Xfce, Mate, and probably other smaller desktop environments. We have very up-to-date packages of Chrome and Firefox that we’ve hardened. The portable parts of Wayland have made (or are making) their way into the ports tree. None of this would be available if there weren’t a bunch of people using it on their desktop.
Why?
It comes with an X server, an incredible array of software, both GUI and terminal based applications that I can install. For my needs OpenBSD is a very capable desktop, and more responsive and flexible then the Windows desktop that work gives me.
I think this title would benefit from the clarification “[…] on my personal laptop”.
!!!
For comparison, ext3, with journaling, was merged into Linux mainline in 2001.
I developed embedded devices whose bootloader read and wrote ext4 files, including symlinks. There is really no excuse not to have a journaling file system on a system that’s larger than a fingernail.
For OpenBSD it may be as similar reason as why they got rid of Bluetooth. Nobody was maintaining that code. It got old, stale. So they got rid of it. I generally actually like this approach, sadly you lose functionality, but it keeps the entire codebase “clean” and maintained.
My guess is that they need people to work on the FS issue. Or, rather they don’t have anyone who cares enough about it onboard to actually write the code in a way that is conductive to OpenBSD’s “style”. Could be wrong, but that’s my assumption.
For comparison, FreeBSD merged soft updates (a variant of journaling) in the 1990’s, and in 2008 announced ZFS support.
OpenBSD had soft updates, but they recently pulled it.
The pragmatic thing to do would be to grab WAPBL from NetBSD since NetBSD and OpenBSD are still relative kin. Kirk McKusick still maintains UFS on FreeBSD so the SU+J works well there but it would be a lot of work to pull up OpenBSD’s UFS.
IIRC WAPBL still has some issues and is not enabled by default. Its primary purpose is not to make the filesystem more robust but rather to offer faster performance.
I’ve never experimented with SU+J, but I’d like to hear more feedback on it (:
I’m not sure how deep WABPL goes but a journal helps you to close some corruption events on an otherwise non-atomic FS by being more strict with the sync and flush events and without killing performance. You can also journal data which is an advantage of taking this approach, although I don’t know that WAPBL offers this currently. Empirically NetBSD UFS+WAPBL seems fairly reliable in my use.
SU+J orders operations in an ingenious way to avoid the same issues and make metadata atomic at the expense of code complexity, the J is just for pending unlinks which otherwise have to be garbage collected by fsck. A large, well known video streamer uses SU+J so it is well supported. Empirically SU+J is a little slower than other journaling filesystems but this might be as much implementation and not algorithm.
Thanks for the feedback. I’ve been wanting to check out SU+J for a while, you got me hyped to dig into the concepts and the code!
Re WABPL: an interesting thread on netbsd-tech-kern
Nice, I had not seen that thread. I think the data journaling they are discussing would be important for OpenBSD as for instance on FreeBSD UFS is used in specific scenarios like embedded devices or fail-in-place content servers and ZFS is used anywhere data integrity is paramount. WAPBL was created in response to the complexity of SU+J as it seems OpenBSD was also impacted by. For that and easier code sharing I would be inclined to go the WAPBL direction, but there may be other merits to the SU+J direction in terms of syncing FFS and UFS against FreeBSD.
OpenBSD FFS has soft updates since a very long time too.
Soft updates have been removed in Feb 2024: https://marc.info/?l=openbsd-cvs&m=171489385310956&w=2
This is really surprising. I thought this was one of those instances of OP “handling things wrong”, but actually it doesn’t seem OpenBSD natively supports anything else than FFS and FFS2 (and without soft updates, like above).
I use OpenBSD on both my personal X220 laptop and my ThinkCentre Tiny desktop. Love the experience on both of those machines. I do see the pain points brought up by the author though (lack of bluetooth support specifically). Luckily, those aren’t deal breakers for me - but I completely understand.
I found myself gravitating to Debian unstable on most of my personal laptops. It offers a reasonable tradeoff of “industry standard” and not being frustratingly out of date without using a boutique OS (I’ve used Gentoo and Arch in the past which can be fun but also consume a lot of time) or quasi-commercial OS (Ubuntu, Fedora, OpenSUSE) with the regular rug pulls and weird forced decisions. I’ve never really been drawn to the different takes on state management or partitioned security OSes and Debian lets me focus on getting whatever else done.
I use FreeBSD on a variety of systems and it doesn’t suffer from these same drawbacks, especially on servers, but it has others.. no current wifi standards support, suspend/resume is very hardware dependent, and graphics are always a sudden unexpected learning event. If you have a desktop where standby doesn’t matter, the official nvidia driver is very stable and performant and I typically use Ethernet - I do my best work on this system.
I have some NetBSD systems (in particular a T480) and really like it but it needs more developer help to offer a compelling mobile option. I do run a couple OpenBSD as well but wouldn’t want to use it for a desktop for performance and filesystem reasons.
Leaving hardware support aside, what would draw you to run FreeBSD instead of a minimal Linux distribution?
I’ve never used BSD extensively, but it is an option I am looking at now for a new workstation I am building.
I’m pretty wrapped up in FreeBSD so there will be some implicit biases for a variety of reasons.
If I use a five dollar word, I would say FreeBSD has much better orthogonality than any Linux distro (although some approach it). Enough of this is cultural that it isn’t an accident, but to borrow a term src is a “monorepo” (the base system is all in one) and this causes some natural alignments between the build system, compiler, libc, administration utilities, man pages and kernel. There seems to always be some low temperature debate on what should be in there, for instance maybe it doesn’t make sense to have the compiler, but having the kernel, libc, man and base utilities in one is a big benefit versus Linux. It makes evolving interfaces a lot more natural, and yet FreeBSD still provides excellent backwards compatibility to at least 4.x.
What this means practically is that if you used a FreeBSD system 20 years ago and today there would not be a lot of surprises, and mostly pleasant ones like a better filesystem (ZFS) and block abstraction (geom) now being the norm.
For system development, it is really low friction versus others. Especially the Linux kernel which intentionally moves and breaks interfaces all the time to make out of tree development painful.
I benefited greatly from getting involved in FreeBSD, even when I was working strictly on Linux systems, so I would recommend it to anyone that is interested in operating systems. It will channel you in different directions than the loudest norms today like Docker and Kubernetes, although there is no technical reason these can be implemented directly or analogously.. and this isn’t always a bad thing. Keeping an open mind and noting what is done well or what is lacking will help in both directions when comparing or deciding when to use what.
Thanks for your detailed explanation.
In terms of security, do BSD Jails offer a better solution to isolate applications?
Linux is a bit behind in this respect, and the X Window System makes the problem worse by essentially letting any graphical application to see key events from other applications or inject spurious ones.
Linux has so much, it is hard to pin down any fair comparison. I view jails as more of an administrative concern, like Linux namespace+cgroup combo, and not a hard security feature although these things don’t hurt to help separate concerns. Landlock is a compelling option on Linux and is inspired by FreeBSD’s capsicum (for some reason, when Google sponsored a direct port, that didn’t get in) – for something like a server process or even a browser I think these are good enough for the masses.
If you really care about isolation the Qubes approach is a bit more plausible because a virtual machine, even though an immense abstraction, has a much easier to reason about boundary in the hardware than any pure kernel approach. The CHERI project, which is FreeBSD adjacent, aims to amend this extreme by moving capabilities into hardware - which might make you trust a kernel construct like jails more.
X has all the capabilities to block this (this is the difference between
ssh -Xandssh -Y; -X uses an “untrusted” connection which puts it in an isolation group, -Y doesn’t, and there’s more beyond that which is even harder to use like the access control extension, or outright running nested servers for even more), it just isn’t turned on typically. Some distros even configure it out entirely in their builds. But if you really wanted to, there are options available.Not an expert on this, but AFAIK you need to run a pretty convoluted jailing setup with applications inside their own isolated X11 servers: https://wiki.gentoo.org/wiki/User:Sakaki/Sakaki%27s_EFI_Install_Guide/Sandboxing_the_Firefox_Browser_with_Firejail
That is probably the strongest solution - that article mentions a couple of the other options but states the nested servers is the tightest barrier, which I’d agree with. Though interesting to note that it then punches holes in that barrier anyway in the name of convenient usability, like the clipboard. That’s the problem with these isolation schemes - you usually do end up poking a lot of holes in it anyway since a fully isolated thing is a pain to use.
I’m honestly surprised you said Debian unstable. Do you run into many issues with broken packages or broken upgrades?
I’ve used Testing for about 15 years, but the few times I’ve tried unstable ended up with broken stuff at the worst possible times. The only problems I have with testing involve proprietary firmware (thanks Apple and Nvidia…) either moving around or dropping support for things.
I have not experienced an unusable system in 2 years. The case for unstable is that testing does not receive any security priority so you can be running vulnerable versions longer. Investing some time into setting up Btrfs snapshots would probably be wise but I haven’t done so myself yet.
I check
apt dist-upgradefor anything peculiar before accepting, in particular there are sometimes days or even weeks where a dependency chain may be broken. An ’apt upgrade` instead will usually work around that until it is fixed. This cycle towards 13 has had some big events, like 64-bit time, rearranging kernel firmware packages, and Qt changes.. Plasma 6 might be a big one too. I would expect future cycles to be a bit more tame t64 was a big one.On Qubes OS: I used it for several years, and really liked it. Though it was always a little bit of a performance slog, especially because I abuse my laptop’s resources a lot (mostly with way, way too many open browser tabs, but also with too many qubes) and hit a lot of bugs because of this. But the real problem was that it was bad for my mental health: I found it turned my professional paranoia up to a pretty extreme level that was mostly worth the tradeoff, but clearly a bit detrimental. Simple operations like copying and pasting all of a sudden became fraught, because I’d ask myself questions like “is it really worth it to copy and paste this text from a less-secure qube into a more-secure qube, or is it short enough that I can just type it out again and avoid an escalation vector?”
I switched to Fedora Silverblue “just to try it out” (I really did intend to only try it for a little bit of time) and haven’t looked back since.
I enjoy reading Solene’s articles, because she always takes a pragmatic, but with an eye on security, approach to things.
I especially enjoyed reading Solene’s articles on OpenBSD, as she was amongst the very few writers out there that discuss about OpenBSD from a non-server point of view.
Thus, I am very sad to read about her leaving the OpenBSD ecosystem, however, reading her article I couldn’t agree more with most of her observations, especially those that also impact OpenBSD deployments as a service platform:
Granted, the OpenBSD has done wonders with its small and underfunded team. Unfortunately, “the world” wants “shiny” things…
Don’t conflate these two. Fine-grained locking can outperform lock-free data structures in the network stack on low core counts. FreeBSD started moving to lock-free data structures a while ago after doing a lot of profiling here. If you have under 8 cores, fine-grained locking is often faster. For 8-16, it’s often not clear which is a winner. Above 16, lock-free can be a win and becomes bigger as core counts scale (data based on decade-old hardware, might be different now).
Unless you’re using in the 10Gb/s range or above or have an lot of concurrent connections, and you’re using a lot of cores for networking, you probably won’t see a big speedup from moving from fine-grained locking to lock-free approaches.
That said, OpenBSD was quite late moving away from a single giant lock. This just scalability problems around 4 cores. I remember a presentation from some OpenBSD folks at a conference 8-10 years ago showing that OpenBSD’s pf was faster than FreeBSD’s. This irritated a bunch of FreeBSD network folks because the presenter didn’t mention that it was on a single-core Soekris system. Adding more cores gave an almost linear speedup in the FreeBSD version (at least up to 8 cores), whereas it gave a slight slowdown on OpenBSD.
I agree that for most “normal” scenarios, the network related performance overhead shouldn’t count much, because the majority of the time would be spent in doing template rendering or waiting for the DB.
But, for my particular case which was network intensive, the overhead would have been felt. Especially since my initial scalability plan resorts much on vertical scalability (i.e. get a bigger VM / box).
I think I wasn’t clear. I am trying to say that ‘fast’ and ‘lock-free’ are not the same thing. If you have a single core spending close to close to 100% of its time in the network stack, coarse-grained (a.k.a. Giant) locking is probably fine. If you have 2-4 cores in this state, fine-grained locking will probably be the fastest approach, assuming that the locks are in sensible places.
Up to about 16 cores, it’s hard to predict and over 16 cores you’ll probably see a speedup from lock-free data structures, thought this isn’t guaranteed (and with a SmartNIC that supports multiple queues, you may see better performance with simple per-core flow routing as some userspace network stacks do, with no cross-core communication at all in the network stack other than for marking in-use ports).
Way back when WhatsApp first announced that they were serving over a million active connections from a single FreeBSD server, the network stack was using fine-grained locking. When Netflix first announced that they were saturating dual 10 GigE links with TLS on a single FreeBSD machine, the network stack was using some lock-free data structures (and some fine-grained locking) but the big speedups came from the combination of KTLS and
sendfile/aio_sendfile.The OpenBSD network stack may not scale in the direction that you need, but that isn’t necessarily to do with the lack of lock-free data structures.
This isn’t recent, it’s just been going very slowly over many years because only one or two people are working on it.
What is lacking in OpenBSD’s Rust support? We have up-to-date packages for Rust, Go, Crystal, all the Ruby versions, Zig, Hare, and probably some other trendy new languages that I don’t recognize in a quick
git logof/usr/ports/lang. Usually if we have a package, it works properly just the same as on other platforms.Indeed, OpenBSD does technically support Rust, as in having a (ports) package for it. (And it seems today it’s even up-to-date, since two years ago when I checked it was a few versions behind.)
However, at least in the Rust ecosystem, having “support” implies having at least
rustupworking for a given platform. Why?rust-upsupport, a developer on another platform can, if not build and link, then at least check if the application would compile for OpenBSD;Here are a few reasons, cited from the
rustupproject, on why it doesn’t yet support OpenBSD: https://github.com/rust-lang/rustup/issues/2168#issuecomment-678608631 – mainly lack of binary bootstrapping of OpenBSD binaries from Linux, and lack of OpenBSD compatibility between versions;Furthermore, when I tried to build my Rust project on OpenBSD (after installing the toolchain from the ports), although I didn’t have too many dependencies, it did fail to build due to some C-based dependency. And, I’m certain if I would have made that work, I would have stumbled upon the next one, etc. Why? Because very few packages are actually tested against anything else than Linux / OSX and perhaps Windows.
Now, I’m not criticizing the OpenBSD developers for this, because in the end it’s the Rust project’s (or community’s) job to make sure their language works on a certain platform they want to support. And the same for various libraries.
However, given this is a recursive / feedback-loop type of problem (i.e. libraries don’t get tested, because language is not supported, because there aren’t much libraries that work, loop), the chain must be broken (or started) somewhere, and I would say that if one at least can
cargo check --target x86_64-unknown-openbsdtheir code against OpenBSD it would be a huge win.For example, in one of my projects, I can build (as in link an executable) at least for Linux, Android, OSX and Windows (all x86-64 and aarch64), plus WASM32-WASI, and although I can’t yet build a FreeBSD binary, I can at least check against it, so that if one wants to build my tool on FreeBSD, some work would have been already done.
If I want to test my code against OpenBSD, I can’t without also running OpenBSD myself.
I don’t think this is an unreasonable ask. I wouldn’t feel confident shipping something I don’t have the ability to test myself.
Sure, you are right, if I don’t also run OpenBSD, I can’t make sure that my tool actually runs on OpenBSD.
Then, if I also don’t run FreeBSD, I can’t make sure that my tool also runs on FreeBSD.
(Perhaps I should also try NetBSD? Or perhaps my time is best spent for DragonflyBSD?)
Then, if I’m at it, if I don’t run OSX, then I can’t make sure that my tool runs on OSX. Wait, there are two main variants now? ARM and Intel based ones? Well, I’ll now have to buy myself two $1K+ laptops to make sure my open-source project works on OSX.
So, now that I’m a couple $K into it, why not add another $K to buy a machine able to run Windows 11, so I can make sure that my tool works on Windows. (What, Windows also has an ARM variant now?)
Or, the alternative where I save a couple of $K just for the hardware, I just make sure it works on my current Linux distribution, and whomever wants to use it on OpenBSD (and other OSs) can just send back bug-reports if it fails to build or run.
I’m sorry for the sarcasm, but I wanted to convey that OpenBSD isn’t the only OS out there, and one can’t run natively all targeted operating systems without eating a lot of energy (and possibly $) and time, that could have been better spent improving one’s project.
Thus, let’s meet half-way: I
cargo checkfor many platforms (this ensures that the Rust code is portable), and if the target has the cross-build tooling for Linux (which OSX, Windows and Android does), I’ll also build it.And if I
cargo checkmy tool, which has (as is the Rust way) a few tens of dependencies, if anyone of them also fails I’ll open bug reports on their projects, thus just one tool checking for OpenBSD translates to multiple projects being also checked.Far from unreasonable, really, though I note that it’s still in the sarcasm section of your post.
That statement of mine was more “in between” sections, and although not unreasonable as you say, is what actually happens in open-source nowadays: most developers just test and run their code on their setup
Which actually means exactly one data point from the cartesian product of (kernel(*version?) * distribution(*version?) * processor(*generation?) * c-library(*version?) * compiler(*version?)).
Thus, in reality, a developer can only actually “hope” that his code actually runs on the myriad possible combinations out there.
When you put it that way it’s pretty surprising how often it works out
I think it is, at least partially. OpenBSD deliberately refuses to have not only a stable OS ABI (which is fair, most OSes other than Linux don’t) but also a stable API. They reserve and make use of the right to break libc headers, even in minor releases.
This makes it impossible for third-party software like Rust to target OpenBSD generically - you need to port it to every new OpenBSD minor release. And I’m not making this up, here are OpenBSD devs asking Rust people for this exact thing and getting told essentially “sorry, no, you’re not special”: https://github.com/rust-lang/libc/issues/570.
Edit: To clarify, I’m not saying OpenBSD are “ wrong” here, just making the tradeoff clear. They are choosing the simplicity and maintenance ease of not having to deal with backcompat headaches over good support for third-party software.
Does OpenBSD even have minor releases? As far as I can tell they just increase the version number by 0.1 every six months, and the number doesn’t indicate anything about what’s changed other than the date.
Correct.
I am curious about that. I think OpenBSD might have never crashed on me. Windows (also post-XP), macOS, Linux, FreeBSD, NetBSD, DragonFly (though that was hardware related) all did multiple times though.
Is this due development? Is this hardware related?
Just surprised about that particular one. All the others feel like “sure, if that’s what you want then OpenBSD is probably not a good choice”.
Very similar experience. Don’t think I’ve ever had a crash. Her experience might also have something to do with solene being an OpenBSD dev, so testing untested stuff possibly leads to some I stability? That’s an assumption on my part.
I’ve used OpenBSD on my personal laptops (and work when I can) for the last 7 years at least. Installation, configuration and system related operations are a joy.
But the less then ideal docker through vm’s way of doing docker does stop me from using it for many contracts and in some contexts it can be very slow. I almost never have crashes and I dont remember losing any files, though I always have to do fsck when my battery runs out, and that happens pretty often.
If I move on from OpenBSD, the likely suspects are void linux, guixSD or Genode SculptOS.
apmdhas-zand-Zoptions to automatically suspend or hibernate when your battery is running low.I run @jcs shutdownd on my Laptops and I have a cron job to start it on boot and that has saved me my many times…
Wow, I don’t even remember writing that. I just use
apmd’s-Zoption now to automatically hibernate when the battery is super low, with a script in/etc/apm/hibernatethat tries to warn me with Xdialog.Does it also imply that OpenBSD Webzine will be no more? )-:
I too use OpenBSD sparingly, but when I can get away with it, I use it.
Most of the time I’m using FreeBSD, MacOS or Linux. I want to get off of MacOS, but I’m not in any particular hurry. I just worry that Apple will eventually ruin enough that it’s no longer useful as my desktop.
I’ve used OpenBSD as my primary desktop since 2001, but I get Solene’s points, and I would like BlueTooth back again (I don’t have the skills to implement it, and I have enough work arounds that it’s a nice to have…), and over the years I’ve seen crashes, but luckily I’ve not lost any data, and usually I’m back working again quickly, even with an fsck of the ffs. I’ve also never really played computer games, as life is too short as it is, but I have run minecraft servers on OpenBSD and installed many of the games in packages for my kids to play with over the years.
RIP. Solene was among the most capable OpenBSD devs.
Jesus, she’s not dead! I’m sure she’s still a capable OpenBSD dev.
I didn’t mention she was dead. But she’s no longer an OpenBSD developer.
I’d love to see your ranking, based on what ?
OpenBSD isn’t even really supposed to be a desktop OS. I’d say it’s more like router firmware. I’m always shocked when someone actually implies they do or have been using it as a desktop OS.
And yes, I know there’s going to be someone who insists they also use it. I’ve also seen people try to use Windows XP x64 Edition well into 2014. Trust me, I have seen no shortage of questionable life choices.
The author of this was previously on the OpenBSD development team. OpenBSD devs tend to dogfood their own OS, so of course she would have used it as a desktop.
This isn’t really true. A few porters do huge amounts of work to keep (among other things) KDE and Chromium and Firefox available for OpenBSD users, and not insignificant work goes into making the base system work (more or less) on a decent variety of laptops. It’s less compatible than Linux but for a project with orders of magnitude less in resources than Linux it does pretty good. But I guess we’ve finally reached the Year of the Linux Desktop if we’re now being shocked that someone would have a BSD desktop.
Using a BSD isn’t weird. OpenBSD specifically is a curious choice, though.
Use it if you like it, don’t if you don’t.
I love curious choices though!
I use OpenBSD as desktop OS for the last 10 years.
Good that you tell me that it’s not supposed to be used as Desktop OS. Otherwise, I wouldn’t have noticed!
You jest, but the blog post legitimately contains a massive list of things the author found very useful in Linux that isn’t in OpenBSD.
almost as if different users have different needs
I would say that the vast majority of OpenBSD developers are using it as their primary OS on a desktop or laptop. I am shocked (well not really anymore, but saddened) that developers of other large mature operating systems don’t use it as their primary OS. If you’re not using it every day, how do you find the pain points and make sure it works well for others?
We have reasonably up-to-date packages of the entire Gnome, KDE, Xfce, Mate, and probably other smaller desktop environments. We have very up-to-date packages of Chrome and Firefox that we’ve hardened. The portable parts of Wayland have made (or are making) their way into the ports tree. None of this would be available if there weren’t a bunch of people using it on their desktop.
XXX isn’t really supposed to be YYY.
For your usage, my usage, a supposed general usage or one of my cat’s usage?
Be thankful that enough people made the “questionable life choice” to run Linux as a desktop OS in the 90s.
Why? It comes with an X server, an incredible array of software, both GUI and terminal based applications that I can install. For my needs OpenBSD is a very capable desktop, and more responsive and flexible then the Windows desktop that work gives me.