This is intended to be comfortable to follow strictly, so more likely to be reliable in practice:
I find it less comfortable. Current SemVer is strictly objective - if breaking, then major. Else: if affects externally-visible behaviour, minor; else patch. BreakVer requires me to make a subjective judgement about what constitutes a major breakage - which requires knowing about all my customers - which is, practically, impossible. Plus, depending on my confidence/arrogance/move-fast-and-break-things-iness, I’ll make classifications that do not match up with customers’ expectations.
There’s only 2 types of version bumps: Those that are definitely safe (non breaking) / Those that require checking the CHANGELOG
That is currently true. You’ve just shifted the boundary of specificity - instead of subdividing non-breaking changes, you’re sub-dividing breaking changes. In practice, either:
folks will be appropriately cautious in checking every major or minor change for impact - in which case, they have no benefit from this method
they will be lazy and not check minor changes - in which case some breakages will still slip by, as they do currently, but the software publisher will not even be in-the-wrong and thus, hopefully, correct their behaviour. A situation in which every actor can make rational justified choices and still end up with a bad outcome is not one we should actively try to design.
People strongly resist bumping major version numbers for every breaking change.
My descriptivist take is that SemVer isn’t what the spec says, but what the tooling does with it, and how people are affected by that.
If you bump the first number, people won’t update quickly, and this may be a good thing or a bad thing, depending on what your goal is.
If you bump the other numbers you may or may not upset a lot of people and get complaints.
So you weigh the risk of getting an avalanche of complaints vs how inconvenient it would be for you if users didn’t update your software quickly, and maybe also complained it doesn’t update automatically often enough.
Yes, it is. It doesn’t matter whether the change is breaking “for someone” - it matters that the change is breaking to a public contract. If you take a dependency on behaviour that isn’t an explicit part of the an established contract, you have no right to expect warning when it changes.
Breaking in semvaris objective. It’s defined by the spec as a change in the behavior of your public interface. If you haven’t documented your public interface, you aren’t following semvar.
While this may be strictly true, it also implies that almost no one is actually following semver, which doesn’t seem like a very productive framing to me.
Huh? When the semvar spec says “public API”, I imagine some docs you can read that list all of the functions in a package along with their type signatures and a description of their behavior. Most of the packages you use have this, no?
That’s the package’s public interface. If the package changes one of those type signatures, that’s a breaking change. If it introduces a new function, that’s not breaking. If it makes a change that violates the documentation for one of its functions, that’s a breaking change. If it makes a change to the behavior of a function that’s consistent with that function’s docs… well either that’s not a breaking change, or as is common the function was inadequately documented.
This all seems fairly unambiguous to me, excepting changes to the behavior of poorly documented functions. Am I missing something?
The example I’ve gone round and round a bunch of times with people on is: Go 1.13 introduced the ability to use underscores as grouping separators in integer literals, like 1_000_000 instead of 1000000.
This also changed the behavior of Go’s integer-parsing functions. For example, strconv.ParseInt() suddenly started accepting and parsing inputs with underscore characters rather than returning an error. And the Go team seem to have been aware that there were people whose code was broken by this change, which would be a problem for Go’s claim that there will never be breaking changes ever, for any reason.
Generally people have argued with me that although ParseInt() was a public function, it was somehow underspecified or ambiguously specified prior to Go 1.13 and therefore it was acceptable to clarify its behavior in Go 1.13 by suddenly changing the inputs it accepted. But this just points to the real purpose of SemVer: it’s about protecting the developer of the code from the user of the code, by giving the developer endless subjective loopholes and ways to say “sure, that change broke your code, but it’s still not technically a breaking change”. For example, any function which does not specify up-front the entire set of potential inputs it will accept and the results it will return for them is subject to the ParseInt() style of “oops, we underspecified it” loophole.
Ah, I get it. There are three things the Go docs for ParseInt() could say:
They could positively state what sorts of integers ParseInt() accepts, so that a reader could confirm it would not parse underscores. E.g. giving a regex of accepted inputs.
They could carve out negative space to allow a variety of behaviors. E.g. saying “parses integers in the same way that the Go language does, which is subject to change”.
They could be silent in the matter.
Reading the actual docs, I’d frankly put them in the first case: they state what the accepted integer syntax is, and give examples, and all of this makes it rather clear that underscores aren’t part of the integer syntax, any more than “thirty seven” would be.
But even if the docs weren’t clear, you don’t get to say “LOL no change is breaking change because I forgot to write docs”. That just means you’ve entered a gray area, and you should be cautious about what counts as a breaking change and your users should be cautious about not relying on too much. It should be a “meet in the middle” sort of a thing, not a “how C++ compiler writers interpret undefined behavior” sort of a thing.
tldr; I’m sorry that people are treating “incompletely documented” the same as “explicitly unspecified”, those are very different things.
Isn’t accepting “1_000” in Go source also a breaking change by the same reasoning as it would be for ParseInt? Input that used to result in an error no longer does.
Maybe in some technical sense, but people rely on both positive&negative behavior of ParseInt() (e.g. rejecting invalid user input), but generally only rely on positive Golang behavior. If “this program started to compile when it used to be an error” was considered a breaking change, every change in language behavior would be breaking.
What I meant was that in the real world, it’s very common for an API to be underdocumented, with the result that it’s not well-defined whether a given change breaks the API or not. Like, you can look at certain changes and say “this really seems like it breaks the API,” but the API was defined vaguely enough that it’s impossible to make any judgment like that.
You say “…excepting changes to the behavior of poorly documented functions,” but I think that’s actually a very large category, in practice :-) Obviously there are some libraries and frameworks that take defining their APIs very seriously, but I would guess that the set of libraries that use SemVer is an order of magnitude larger than the set of ones that are strict about it in this way.
Yeah, that all makes sense. I’d argue that if it’s hard to make that judgement call, the library should be conservative and bump the major version number.
Is there a versioning system you think does better in the presence of a poorly defined API?
I don’t know of one, and I suspect that carefully defining the API is a prerequisite for any versioning system to be able to give you the kind of guarantees we want.
Hyrum’s Law describes a common pathological condition of dependency relationships between software modules, it doesn’t define a de facto baseline/standard guarantee or expectation of compatibility.
That person is welcome to be upset and to ask for consideration, but they are by no means entitled to it. SemVer is about breaking the explicit, published contract of software, not about breaking any hypothetical consumer. If you take a dependency on undefined behaviour, you have no rights to complain when it changes, nor any justification to expect warning when it does.
I’m concerned that Bluesky has taken money from VCs, including Blockchain Capital. The site is still in the honeymoon phase, but they will have to pay 10x of that money back.
This is Twitter all over again, including risk of a hostile takeover. I don’t think they’re stupid enough to just let the allegedly-decentralized protocol to take away their control when billions are at stake. They will keep users captive if they have to.
Hypothetically, if BlueSky turned evil, they could:
ban outside PDSes to be able to censor more content
block outside AppViews from reading their official PDS
This would give them more or less total control. Other people could start new BlueSky clones, but they wouldn’t have the same network.
Is this a real risk? I’m not sure. I do know it’s better than Twitter or Threads which are already monolithic. Mastodon is great but I haven’t been able to get many non-nerds to switch over.
Hypothetically, the admins of a handful of the biggest Mastodon instances, or even just the biggest one, could turn evil and defederate from huge swathes of the network, fork and build in features that don’t allow third-party clients to connect, require login with a local account to view, etc. etc.
Other people could start clones, of course, but they wouldn’t have the same network.
(also, amusingly, the atproto PDS+DID concept actually enables a form of account portability far above and beyond what Mastodon/ActivityPub allow, but nobody ever seems to want to talk about that…)
The two situations are not comparable. If mastodon.social disappeared or defederated the rest of the Mastodon (and AP) ecosystem would continue functioning just fine. The userbase is reasonably well distributed. For example in my personal feed only about 15% of the toots are from mastodon.social and in the 250 most recent toots I see 85 different instances.
This is not at all the case for Bluesky today. If bsky.network went away the rest of the network (if you could call it that at that point) would be completely dead in the water.
While I generally agree with your point (my timelines on both accounts probably look similar) just by posting here we’ve probably disqualified ourselves from the mainstream ;) I agree with the post you replied to in a way that joe random (not a software developer) who came from twitter will probably on one of the big instances.
For what its worth I did the sampling on the account where I follow my non-tech interests. A lot of people ended up on smaller instances dedicated to a topic or geographical area.
While it’s sometimes possible to get code at scale without paying – via open source – it’s never possible to get servers and bandwidth at scale without someone dumping in a lot of money. Which means there is a threshold past which anything that connects more than a certain number of people must receive serious cash to remain in operation. Wikipedia tries to do it on the donation model, Mastodon is making a go at that as well, but it’s unclear if there are enough people willing to kick in enough money to multiple different things to keep them all running. I suspect Mastodon (the biggest and most “central” instances in the network) will not be able to maintain their present scale through, say, an economic downturn in the US.
So there is no such thing as a network which truly connects all the people you’d want to see connected and which does not have to somehow figure out how to get the money to keep the lights on. Bluesky seems to be proactively thinking about how they can make money and deal with the problem, which to me is better than the “pray for donations” approach of the Fediverse.
Your point is valid, though a notable difference with the fediverse is the barrier to entry is quite low - server load starts from zero and scales up more or less proportionally to following/follower activity, such that smallish but fully-functional instances can be funded out of the hobby money of the middle+ classes of the world. If they’re not sysadmins they can give that money to masto.host or another vendor and the outcome is the same. This sort of decentralisation carries its own risks (see earlier discussion about dealing with servers dying spontaneously) but as a broader ecosystem it’s also profoundly resilient.
a notable difference with the fediverse is the barrier to entry is quite low
The problem with this approach is the knowledge and effort and time investment required to maintain one’s own Mastodon instance, or an instance for one’s personal social circle. The average person simply is never going to self-host a personal social media node, and even highly technical and highly motivated people often talk about regretting running their own personal single-account Mastodon instances.
I think Mastodon needs a better server implementation, one that is very low-maintenance and cheap to run. The official server has many moving parts, and the protocol de-facto needs an image cache that can get expensive to host. This is solvable.
Right! I’ve been eyeing off GoToSocial but haven’t had a chance to play with it yet. They’re thinking seriously about how to do DB imports from Mastodon, which will be really cool if they can pull it off: https://github.com/superseriousbusiness/gotosocial/issues/128
That’s true, but I’ve been hooked on Twitter quite heavily (I’ve been an early adopter), and invested in having a presence there.
The Truth Social switcheroo has been painful for me, so now I’d rather have a smaller network than risk falling into the same trap again.
Relevant blog post from Bluesky. I’d like to think VCs investing into a PBC with an open source product would treat this differently than Twitter, but only time will tell.
OpenAI never open sourced their code, so Bluesky is a little bit different. It sill has risks but the level of risk is quite a bit lower than OpenAI was.
OpenAI open sourced a lot and of course made their research public before GPT3 (whose architecture didn’t change much[1]). I understand the comparison, but notably OpenAI planned to do this pseudo-non-profit crap from the start. Bluesky in comparison seems to be “more open”. If Bluesky turned evil, then the protocols and software will exist beyond their official servers, which cannot be said for ChatGPT.
[1]: not that we actually know that for a fact since their reports are getting ever more secretive. I forget exactly how open that GPT3 paper was, but regardless the industry already understood how to build LLMs at that point.
There are some cases where I do want different commits on the same branch/pull request to run all actions. For example, if I am updating workflow dependencies for a workflow that normally runs on merges to main, I will temporarily make them run on my branch to test/validate. I make two commits—one with the updates, and another that runs the workflow on my branch. Once I have validate the change, I remove the last commit.
I love this opinionated POSIX standard. Instead of baking in a bunch of hacks to support filenames with newlines, they just said “don’t do that, not supported going forward”. That’s a change I can get behind.
what will happen to existing files tho? i have some, currenty all programs support those without any hacks, except one (ls). will i be unable to even rename or delete them if i mount my storage on a new posix 2024 system?
the following utilities are now either encouraged to error out if they are to create a filename that contains a newline, and/or encouraged to error out if they are about to print a pathname that contains a newline in a context where newlines may be used as a separator
I’m curious how this plays with locales. The most common places I’ve seen this issue are where the file is created with something like a Big5 locale and then viewed in a C locale.
Trail byte on Big5 is 0x40 or higher, so not line feed.
It would really make POSIX ready for 2024 to drop non-UTF-8 locales, though. (Which probably won’t happen as long as someone finds it important to be able to claim a Big5 AIX system POSIX-compliant.)
At some point I need to write an actual article about this, but I’ve recently been thinking that “blocking” APIs in general are a design mistake: everything is fundamentally concurrent, but OSes use blocking APIs to hide this concurrency, which tends to inevitably break, and mixes concurrency with parallelism. Even something as simple as “spawn a process, collecting both its stdout and stderr” requires proper concurrency. See how Rust’s Command::outputcan’t be expressed using std-exposed APIs.
Which is the smaller problem! The bigger problem is that we don’t actually know how to write concurrent programs. This is an unsolved area in language design. And the systems reason for why the problem is unsolved is, thanks to the existence of blocking APIs, languages are allowed to shirk here, moving what is fundamentally a language-design issue to the OS-design.
IO-uring (and previously, hilariously, JavaScript) models the problem of concurrency more directly, without introducing a midlayer. Hopefully it will create enough of evolutionary pressure for language design to get the problem solved finally.
There was an interesting period about 25 years ago, when the web was growing fast and really stressing the capabilities of the hardware and operating systems of the time. This is when the c10k problem was identified. The performance pressure led to a bunch of interesting operating systems research.
I’m pretty sure I read a paper around that time which described a kernel/userland API with a design roughly along the lines of io_uring — though its point of comparison was hardware interfaces with command queues and DMA buffers. I had a very vague memory of Vivek Pai’s IO-Lite but that has a different shape so there must be another paper I’m failing to remember.
I wondered if a fully asynchronous kernel API could solve the M:N threading problems that were happening around the same time. But that whole area of research dried up, as the problems were dealt with by sendfile(), 1:1 threading, and computers getting faster. Boring!
It was really hard to handle 10,000 concurrent connections from one server in 1999. The challenge was to make it easier, to make the kind of performance engineering that went into cdrom.com available to a lot more systems.
IIRC, the fundamental challenge is handling 10K long-running connections that are not very active. If each of those connections is actually producing significant load on the hardware, either CPU load or I/O load, then the hardware can’t handle 10K connections anyway, because we have neither 10K cores nor 10K independent I/O channels.
I found a copy of the SEDA SOSP paper which is a bit shorter than Welsh’s PhD thesis :-) It seems to be about getting more concurrency out of Java, not about changing the userland/kernel API.
It’s a compelling argument, but I think in some ways async APIs end up constraining the design space of the framework/OS/whatever offering it. For example, QNX’s main message passing primitives are blocking so that, on the fast path, context is switched to the callee without a pass through the scheduler.
You absolutely hit the nail on the head. Given how absolutely fundamental asynchronous computation is to how computers work, it’s surprisingly underexplored.
I think this is partly due to a decline of groundbreaking OS research, and partly because there are barely - if any - modern languages purpose built for doing deep, systems research. The problems become apparent almost instantly. At the latest when you’re a few hundred lines into your first kernel, you’ll be presented with DMA interfaces for all kinds of peripherals. What now?
This is not the kind of problem that most language designers are facing. Most PLs are made to solve application-level problems, instead of having a principled, hardware-first approach to language design. I think Rust with embassy-rs has been solving some of these challenges in a pretty nice way, but it shouldn’t end there.
I agree on the lack of OS research. Mothy gave a keynote at SOSP a few years ago pointing out that twenty years ago there were multiple new kernel papers each year at OSDI / SOSP whereas now we average less than one. I’m slightly biased, because the last paper I submitted with a new OS design was rejected on the ground that we didn’t do a performance comparison to an OS that required ten times as much hardware and that we were possibly vulnerable to an attack that, by construction, cannot happen on our system. Ho hum.
I disagree on language research though. There’s a lot of interesting work on asynchrony. Most PL research takes at least ten years to make it into mainstream programming languages. Verona’s behaviour-oriented concurrency model is designed to work directly with things like DMA and we’ve ported it to C++, Rust, and Python. It works best with a memory management model that puts object graphs in regions and allows them to be atomically transferred, which depends on a type system that can do viewpoint adaptation. We’re not the only people working on things like this and I expect mainstream languages in the 2030s to have a lot of these concepts.
Mothy gave a keynote at SOSP a few years ago pointing out that twenty years ago there were multiple new kernel papers each year at OSDI / SOSP whereas now we average less than one.
That’s brutal. Too brutal in my opinion to only come from spurious rejections. From the outside, I would suspect the main drive is that nobody makes kernels any more: apart from specific niches like real time or formal verification, the world has basically been taken over by NT, Linux, and XNU. And a major reason behind such concentration is that every kernel needs to have dedicated drivers for the insane diversity of hardware we use — except NT, hardware vendors being business savvy enough to write the driver for us. (See The 30 Million Lines Problem.)
Those three kernels are big. Making any addition or change to them is likely a significant endeavour. We likely need the ability to quickly write practical kernels from scratch if research is to take off again. But no one can possibly write a practical kernel if they have to, let’s be kind, port all the driver code from Linux. So the first step has for hardware vendors to design interfaces for humans, and then give us the manual. But given how suicidal this would be business wise (who would ever buy hardware that doesn’t come with a Windows driver out of the box?), we probably need regulators to step in: say the US, or the entirety of the EU, ban the sale of any hardware for which the same company distributes software — or at the very least, proprietary software. With such a setup, I would expect hardware vendors to quickly converge on relatively uniform hardware interfaces that aren’t stupidly complex.
But then there’s the other side of the fence: user space. Since the ossification of OSes we came do depend on a huge stack of software that would take significant effort to port anywhere not POSIX-ish. And that’s for the stuff we have the source code of…
I don’t really think that’s true. For a research project, you need a few drivers and most people just pick up the NetBSD ones (the RUMP kernel work makes this trivial). There’s a big unexplored space for both big and small operating systems. With a very small team, we were able to write an RTOS that has a solid FreeRTOS compat layer and can run most existing embedded software (source compatible) and we were designing and building the core that ran it as well. No one has really built an OS for cloud computing, but it could remove a lot of stuff that desktop operating systems need (no IPC or storage layers, since those will both sit on top of a network).
fwiw I think existing unikernels fit that bill (e.g. unikraft, nanos). They mostly target virtualized hardware like KVM, and seem pretty focussed on the cloud use-case (fast boot, small images, and efficiency). Being in KVM-land certainly removes a lot of implementation complexity, since bringup and platform init is mostly done.
I disagree. I think unikernels are applications for cloud computing. They’re hampered by the fact that the OS that they run on (Xen, Hyper-V, Linux) is not designed for the cloud. They have network devices, for example, but a good cloud OS would make it easy to take advantage of hardware TLS offload and terminate connections directly in a unikernel.
Unikernels have had almost no commercial adoption precisely because there isn’t a good OS to run them on. The benefits of unikernels are incompatible with the billing models of cloud systems. A unikernel invocation may consume a few MiBs of RAM and a few CPU seconds (or hundreds of CPU milliseconds). The operating systems that run them are not set up to handle this at all. They are intended to account for CPU usage in minutes and RAM in GiBs.
If you’re running a unikernel in KVM, there’s little benefit in it over just running a Linux process. You then get to share a TCP/IP stack and so on with other processes and amortise more costs.
Ah then I misunderstood your original comment. So it sounds like you’re unhappy about the currently dominant hypervisor abstraction, that I would agree with. The interfaces for efficient resource sharing are pretty borked.
A unikernel invocation may consume a few MiBs of RAM and a few CPU seconds
Are you talking about serverless here? Unikernels can be long-running services, at least that’s how I’ve run them in the past.
For me, the big benefit of unikernels is their ability to scale down. If you’re handling sustained throughputs of thousands of network messages per second or more, there are some benefits, but if you’re implementing services that have long idle periods of no activity and bursts of very high usage, unikernels’ abilities to either exit completely and then restart in one network round trip time or scale back to almost no resource usage is a huge win. Cloud billing systems are not set up for that at all.
I don’t really like the term serverless, because you don’t get rid of servers. The way most FaaS services are implemented today comes with ludicrous amounts of overhead. Each service starts in a lightweight Linux VM, that runs an OCI container, that runs a language runtime and some support code, which then runs the customer code. We estimated an efficient system with good hardware-software co-design to do build an attestable base with TLS flow isolation to individual services would allow at least an order of magnitude denser hosting and provide better security properties.
Thanks for the link. In your view, would the BoC as a concurrency paradigm allow for a competitive implementation for e.g. performance-critical embedded systems? Are there places where you’d say “this place needs an escape hatch to close the last few percent perf gap” or so?
I’m genuinely interested, because I was looking for things beside stackless async/await that would work well for highly constrained environments.
Possibly. BoC is really equivalent to stackless coroutines, but with a coordination layer on top and an ownership model that makes it safe. Each when captures the things it need to complete and runs without the parent’s stack.
The problem for BoC on resource-constrained systems is that any behaviour can spawn arbitrary numbers of other behaviours, which can exhaust memory, so you’d need to be a bit careful. It’s probably no different to avoiding deep recursion though.
You definitely wouldn’t want to use the Verona runtime for small embedded systems (it’s nice in a kernel though. I ported it to run in the FreeBSD kernel a few years ago) but I have been pondering what an embedded BoC runtime would look like.
I think this is partly due to a decline of groundbreaking OS research
I mean, utah2k was written nearly a quarter-century ago. (There is a certain irony of someone from Bell Labs writing this, considering the output of Bell Labs doing a lot to slow systems research…)
In the 1970s, this was addressed.
Quite a few popular OSs back then treated all base I/O calls as async.
Blocking was a unix thing.
Out of necessity, the Berkeley unix apis on unix tried to walk that back. (They also hacked a few other unix design choices)
Programmers on those older OSs successfully built multiuser database and timesharing systems.
For simple programs, many liked the simplicity of the unix style,
IMNSHO, the underlying problem is that we decided a long time ago that the procedure call (subroutine call, function, method, etc.) is our fundamental model of abstraction. Not just for interacting with the operating system, but pretty much for everything. (“Decided” might be overstating it…it just sort of happened for various reasons, many good at the time).
I’m not 100% convinced that concurrent-only is the way to go (and I say this as a JS dev who also uses Javascript as a good example of what concurrent-only could look like). But I agree that most languages need to choose one or the other. You either go all in on concurrency (like in Javascript, where synchronous APIs are rare and usually only exist a special-case exception alongside an equivalent async API), or you accept that concurrency is always going to be an opt-in feature with less support that will require its own way of doing things.
I’ve seen this a lot in Python, and I think we’re seeing it now again in Rust, where async support gets added, and creates a split ecosystem. You can’t mix and match, so you need to decide: am I using the sync APIs or the async ones? I wonder if there are other concurrency ideas that could mitigate this somewhat, but I think it’s just a fundamental divide. Reading files, interacting with sockets, executing processes etc are fundamental OS-level tasks, and deciding how you’re going to do those cuts to the core of a language’s standard library. In Rust, I know that some of the big proponents of the Async Way are aware of this problem, but I think their goal is to push async as the predominant way, and I’m not sure Rust is set up for that yet.
(As an aside: I know C# also was synchronous and then added the async keyword - are there similar issues there? Is there a divide between the sync and async worlds?)
The thing is, you can’t choose only one! Here’s a mundane, boring example:
You spawn a process. You want to collect its stdout and stderr. You can’t do just
let stdout_bytes = child.stdout.read_to_end();
let stderr_bytes = child.stderr.read_to_end();
because this might deadlock. The child process might interleave printing to stderr&stdout, so you have to interleave reading them. What you want to do here is to issue both read syscalls concurrently and wait for one of them to finish. So you end up writing platform specific code like this:
Another case you always hit when programming mundane things is that something does a read call, and you really want to “cancel” that read (even if it currently blocks). This again necessitates pretty horrific work-arounds with installing a process-global signal handler: https://github.com/rust-lang/jobserver-rs/commit/4c8a5985e32fa193350f988f6a09804e565f0448
In the opposite direction, DNS (and, until io_uring, even file IO) — there’s just no non-blocking APIs for these!
So, I don’t think it’s up to the languages to fix this, really. It’s the OS which first needs to provide reasonable API. Only then can language designers of production languages figure out how to express the concurrency in the source code (to echo David’s comment, I suppose we might have figured how to deal with concurrency in academia, but this certainly haven’t quite percolated to day-to-day languages. Go & Rust seems like they are improvements, but also both feel quite far from optimal).
There are plenty of nonblocking DNS APIs. The gap is in the higher-level name service APIs that bundle up lots of non-DNS name databases, such as the hosts file, mDNS, NetBIOS, NIS, etc.
You can’t perfectly choose one, but I think you can abstract surprisingly well by choosing one route or the other. As you point out yourself, we’ve done pretty well pretending that things like file IO (and even memory access) are synchronous, idealised things. Yes, there are weird edge cases where the abstraction breaks, but most of the time it holds surprisingly well.
Going the fully async route brings its own problems, but it’s still in many ways just a different abstraction. Many async libraries handling file IO just run synchronous APIs on a different thread, because it’s simple, it works, and we still get the concurrency we were expecting.
So I think a lot of this is about the abstractions we chose to use on top of these sorts of APIs, and I think language designers have a lot of leeway there. Yes, the OS APIs will affect those abstractions - what works well, what works efficiently, what leaks constantly into the abstraction layer etc - but there is no shortage of interesting approaches to concurrency on top of largely synchronous APIs.
Many async libraries handling file IO just run synchronous APIs on a different thread, because it’s simple, it works, and we still get the concurrency we were expecting.
Until recently, Linux didn’t have usable async file IO, so you had to do that.
I didn’t consider the implication that VI is as bad as those other things, it’s not. Whatever editor makes you happy, use it.
If it helps, I use vim mode. I just think others might not know that “Dave likes X” is associated with some ideology they might not agree with. I should have spelled out that intention better.
Starting with the first post, it reads like a freedom-of-speech opinion more than “liking” Andrew Tate.
That’s a trolling, to get someone to point that out. The other links aren’t that different.
But what’s the ultimate purpose?
I really like how this site leans into the actual tech, and you typed up the kind of list that adds nothing to the tech merits of TFA, like discussing LazyVim would.
I think if you want to start cancelling someone you disagree with, using strawman lists or not, there’s X/Twitter for that.
Your statement about adding Vi to the list isn’t a troll or a joke, it’s an excuse, or so it reads.
The Bozo Bit is a hallowed tradition in hacker culture. I think it’s relevant to remind people that they might want to set the Bozo Bit on the author of an article.
The article is about the opinions of someone in tech. Sharing their other opinions is very relevant IMHO. If OP only wants to talk about VI without the association of an author they have plenty of options to do that. Sort of like how if Dave ONLY wanted to talk about freedom of speech, he could have chosen from 7.951 billion people that aren’t Andrew Tate.
I think if you want to start cancelling someone
I don’t think that stating things people have said publicly while citing them is “cancelling” them. I feel people use the term “canceling” when someone states an observation they don’t like. Not to mention the vast majority of “canceled” people are in fact still quite influential and very much around. I appreciate you think my words are so powerful, however I an assure you…if I had the power to cancel anyone, I think I would be the first to know.
I like how this site has people discussing things that they find relevant and topical. As you noted, you can easily hide my discussion and move on if you get nothing out of it. But you commented, now again. I like that you got to share your perspective of the situation and I got to share mine. That’s what I want to see in a healthy community. To hear and be heard. That’s all I’m looking for, you don’t need to agree with me, I just want an ACK that I’m heard.
The article is about the opinions of someone in tech. Sharing their other opinions is very relevant IMHO.
How so? That just seems like an attempt to easily dismiss people / ad hominem them. What would his opinions on Tate/Peterson/Shopify hosting Breitbart/DEI have to do with his opinions on Vim?
I appreciate you’re (probably) being facetious and making an editor wars joke when you imply we can safely dismiss his praise of Vim based on his political opinions, but all that does is open up much more contentious political wars for the sake of a joke on mostly-exaggerated editor wars; moreover, a joke that would simply fall flat to anyone who doesn’t agree with the political takes you find so obvious or finds the inverse obvious.
I don’t know the rules around discussion of politics on this site (or baiting people into such discussions), and while I’m very open to having political discussion be allowed, based on my previous experience, I don’t think that’s the kind of site that @pushcx is going for.
Edit:
Sort of like how if Dave ONLY wanted to talk about freedom of speech, he could have chosen from 7.951 billion people that aren’t Andrew Tate.
Defending principles, like freedom of speech, usually entails defending the most unsympathetic people (to any given audience), since sympathetic people usually don’t become victims of violations of that principle. Nobody’s restricting the speech (or even trying) of someone whose words everyone agrees with.
I don’t think that stating things people have said publicly while citing them is “cancelling” them. I feel people use the term “canceling” when someone states an observation they don’t like.
While not exactly censorship, I do think using ad hominems, poisoning the well, and other such responses to get people to dismiss someone’s opinions before hearing them out aren’t exactly conducive to truth-seeking or even to free expression (especially on completely unrelated topics; if he had a track record of dumb technical takes, for example, it’d be more excusable).
Not to mention the vast majority of “canceled” people are in fact still quite influential and very much around.
Agreed, but really, unless you can draw a line from DHH “liking” Tate — which the link you posted to prove that claim doesn’t — to Vim, then it’s off-topic, liable to bait people into further off-topic discussion which are themselves liable to be acrimonious discussions, acrimony that could have been avoided since it’s all off-topic anyway.
Flagging 12 day old comments as “off topic” and misrepresenting their contents isn’t exactly a community high road to be starting from. How did you even find this thread?
Wow, I think I touched a nerve, that’s a lot of deflections. “You’re hardly innocent, you misrepresented me and you’re flagging old content”, oh, and I detect a possible insinuation of brigading.
We can all accuse each other of misrepresenting people; you yourself were (truthfully IMO) accused due to your initial comment of misrepresenting DHH’s posts as him liking certain people, when they were clearly not him expressing a liking, or even support, of those people.
There’s no rule of the site or norm of civility to not misrepresent someone (except intentionally), because who would adjudicate that? I don’t think I did misrepresent you, but even if I did, that’s obviously unintentional, a simple matter of disagreement which has nothing to do with “community high road”.
As for the age of the submission, I don’t think 12 days is that long. I found the thread because I like to check out submissions past the front page, since I don’t check lobste.rs so frequently.
How does my alleged misrepresentation absolve you of the clearly uncivil — and possibly rule-breaking — behaviour of dredging up someone’s irrelevant-to-the-submission political opinions? Don’t try to point the finger back at me because I might have accidentally misrepresented you to deflect from a clearly uncivil post (especially one where you yourself could be credibly accused of misrepresenting others).
I’m not a fan of Jordan Peterson, but all I can gather from that blog post is that David read his book and likes the concept of meaningful responsibility providing purpose in life. He doesn’t appear to be defending him like he does in the Andrew Tate article, just citing him for the term “meaningful burden”. I would personally leave that out of a list like this.
Agreed! I just didn’t feel like it added anything at all here, so I was wondering if I was missing something (like an actual belief that this means vi too is destined to doom, or what).
I don’t think the joke is relating vi to his polemics. I’m guessing it’s a humorous way to bring up to people who are unaware that DHH has terrible opinions (because this article about vi doesn’t really make any of them clear).
I use VIM and VIM keybindings. VI is fine. I was trying to relate my comment back to the original post in which Dave says he likes a thing. Which made me remember some of the other things he likes. I don’t like those, but I like VIM.
Are there publicly available RISCV chips which have or are planning to integrate CHERI? Since this is a MS research project, does Azure plan to roll this out in the long term?
Are there publicly available RISCV chips which have or are planning to integrate CHERI?
Last I heard one will be on the market later this year. I’m sure that once David Chisnall sees this thread he’ll post additional information if there is any.
No one has run CHERIoT Ibex on Altera FPGAs yet, but it works nicely on fairly cheap Xilinx ones. The CHERIoT SAFE project is probably a good starting point.
C sucks, but until I have a working Rust compiler and the time to rewrite a few tens of thousands LoC in Rust, I’m still going to be interested in what’s going on in the C ecosystem.
There are some things that need to be written in a low-level language and so C or C++ (or assembly) remains the best choice. They implement core parts of the abstract machines of higher-level languages and so are intrinsically unsafe.
There are some things that were written in C a long time ago and so would be expensive to rewrite and, because they’ve had a lot of in-production testing, a rewrite would probably introduce a lot of bugs and need more testing.
And then there are new projects, doing purely application programming tasks, written in C. And these are the ones where I wish people would just stop. Even on resource-constrained systems, C++ is as fast as C and lets you build safe abstractions. For larger targets, there are a lot of better choices (often Rust is not one of them: don’t do application programming in a systems language).
The fun you can have with C is the kind that leaves you wanting to take some strong antibiotics when you wake up because you don’t know what you caught.
The fun you can have with Rust is the kind that leaves you wanting to take painkillers because you had to listen to a fanboy ramble about memory and thread safety all night.
Everything was better in the old days. We should Make Programming Great Again. Where are the COBOL people when you need them?
I enjoy programming. It is still great in my opinion. Lot of good programming literature, languages and libraries are free and open source. What not to love.
Eh, I think they’re cute. It’s a shame when C codebases are deployed in any serious context, but I’m always up for curiosity languages/libraries, especially for preserving older computer architectures. We gotta keep this stuff alive to remember where we came from.
Yeah yeah we’ve all seen the white house report. I can’t remember exactly how their guidance was worded, but I would assume it’s more along the lines of “whenever possible, in contexts where cybersecurity is a concern, prefer memory-safe languages such as Rust or Java over non-memory-safe languages such as C or C++” which is actually very reasonable and difficult to argue against.
I think there are contexts which qualify as “serious”, but where cybersecurity isn’t a concern. Also, IIRC there are some (quite esoteric, to be fair) soundness holes in Rust’s borrow checker (I think there’s a GitHub issue for one that’s been open for like 5 years and they haven’t decided on the best way to resolve it yet). Furthermore, rustc hasn’t been formally verified, and then of course there’s always unsafe code where memory safety bugs are still possible. So I think through use of sanitizer instrumentation, testing, fuzzing, etc. you can get a C project to the point where the likelihood of it having a memory safety bug is on par with what Rust can give you. Take SQLite for example—personally I would have as much faith in SQLite’s safety as I would in any other database implementation, regardless of language.
I didn’t really intend to make this some kind of language debate, but if you insist:
So I think through use of sanitizer instrumentation, testing, fuzzing, etc. you can get a C project to the point where the likelihood of it having a memory safety bug is on par with what Rust can give you.
So you mean… just use Rust. Because Rust is essentially C with all that stuff baked in. That’s kind of the point of it. It’s C with dedicated syntax for verification tools (the ones it’s built with) to verify it, along with some OCaml control flow structures to make code more streamlined. You really can just kind of think of Rust as being C++ 2 (Not C++++ because that’s C# (that’s the pun of the name, if you never knew)).
And I’m not sure why people are still doing this “well the compiler might have a bug or two and you can manually mark things as unsafe so that means its as insecure as C” reach, it really doesn’t make any sense and nobody is accepting that as a valid argument.
You keep saying that Mike, but I don’t think you quite grasp the consequence, it’s much bigger for Ruby than for Python, because you need to add a lock to all mutable objects, and in Ruby strings are mutable…
Also I’m yet to see proper benchmarks of the single threaded performance impact. They think they can reclaim the lost performance elsewhere, but that’s because Python has refused any complex optimizations for a long time, that’s not the case of Ruby.
Finally, Python’s multi-processing story is quite bad because ref counting is nasty for CoW, that’s not the case of Ruby.
I know, they said multithreading in Ruby was impossible because the ecosystem wasn’t thread safe. Sidekiq was the tool which forced the community (including Rails) to take multithreading seriously.
How do you eat an elephant? One bite at a time. My favorite timeline has Ruby aggressively adopt frozen strings and deprecate C extensions.
Adopting frozen strings won’t change anything, as long as a string can be mutable, Ruby has to reserve space for a lock in every String and has to check for it. For a language that’s heavily used to render HTML etc, the performance impact would be huge.
As for deprecating C extensions, it just isn’t realistic, and even if it was that wouldn’t be the only issue.
I know you don’t like it, but multi-processing for web servers and job workers works perfectly fine.
One nitpick: a blog post is not a forcing function, no one fixed thread safety bugs because of Yehuda’s post.
I had to create the connection_pool gem in 2011 because Ruby did not have an API for sharing connections amongst threads. AFAIK Sidekiq and its infrastructure enabled the first real widespread usage of multithreading in Rails. YMMV.
a blog post is not a forcing function, no one fixed thread safety bugs because of Yehuda’s post.
I linked to this post because it account of the situation at the time and ends with a good reminder:
Rails 2.3 was threadsafe too.
Rails 2.3 was 2009.
I had to create the connection_pool gem in 2011 because Ruby did not have an API for sharing connections amongst threads.
I’m not sure what’s this is supposed to mean.
AFAIK Sidekiq and its infrastructure enabled the first real widespread usage of multithreading in Rails. YMMV.
No offense but no. Puma predates Sidekiq, and even before that Webrick had multi-threading as an option since ages, as well as passenger (AKA mod_rails or mod_ruby) and Rainbows.
Backward compatibility concern of course but also:
Need to add a lock to all mutable objects, which in Ruby includes strings, which means blowing up memory usage and hitting single threaded performance HARD (It’s still unclear to me how much slower Python is without GIL, but I suspect it’s in the double digit range, would be worse for Ruby).
Python can reclaim the lost performance elsewhere because they only started to prioritize performance recently, so they may be able to remove the GIL and optimize enough to stay at more or less the same performance. Ruby doesn’t have that luxury.
Ruby’s multi-processing story is much better than Python’s, thanks to its GC copy on write works pretty well.
Ruby is predominantly used for share-nothing tasks (typically web) for which multi-processing isn’t necessarily worse than multi-threading. You use a bit more memory, but get isolation and truly lock free parallelism.
The Ruby GC isn’t concurrent, so even without GIL you’d still contend there, whereas Python’s rely predominantly on ref-counting, so less impacted.
The part of the Python community pushing for free-threading is the one that use GPUs, none of that is happening with Ruby.
Ruby is predominantly used for share-nothing tasks (typically web) for which multi-processing isn’t necessarily worse than multi-threading. You use a bit more memory, but get isolation and truly lock free parallelism.
Which by the way has always been the reason for shrugging off GIL in the Python community when it was predominantly a Web language: it just wasn’t problematic. But then science and AI happened :-)
You speak of it confidently so I assume someone actually looked into it. Any chance you could share some links on the topic? Experiment results, discussions, etc.
I understand that setting/reading/waiting on a lock is an overhead but is it 3%, 30%, 300%? How much space does a lock need? Do object with lock require 2x space or is it a single byte or even bit? Can we only lock objects that are accessed cross-thread and mitigate a lot of slowdown?
I can believe that removing GIL is a big and complex task. I can believe it might not be worth the effort. But, TBH, I’m not convinced it’s not something worth looking into. And granted, I’m not the one that will probably do this work so I can’t demand anything but I’m curious, I’d like to know more about this.
I understand that setting/reading/waiting on a lock is an overhead but is it 3%, 30%, 300%?
It’s an hard to answer question because it depends how much a given application would need to lock. But my guess is “double digits”%.
To give you an idea, recently we had an issue in Ruby 3.4-dev where for each allocation we were hitting the ractor path, so had to do things very similar to what we need to do in the absence of GVL, including an atomic increment, and that had a very visible impact on the benchmarks we track at https://speed.yjit.org/. And that was just for allocations, dropping the GVL would mean slowdowns on most method calls.
How much space does a lock need?
In the case of the Python implementation it’s over 20 bytes per object (ob_tid + ob_mutex + ob_ref_local + ob_ref_shared). That’s huge given currently the Ruby object header is 16B.
struct _object {
_PyObject_HEAD_EXTRA
uintptr_t ob_tid; // owning thread id (4-8 bytes)
uint16_t __padding; // reserved for future use (2 bytes)
PyMutex ob_mutex; // per-object mutex (1 byte)
uint8_t ob_gc_bits; // GC fields (1 byte)
uint32_t ob_ref_local; // local reference count (4 bytes)
Py_ssize_t ob_ref_shared; // shared reference count and state bits (4-8 bytes)
PyTypeObject *ob_type;
};
I can believe that removing GIL is a big and complex task.
It’s not so much about the effort, it’s a lot of work but doable. It’s about whether the result is desirable.
The Ruby async/multi-threading story has marked it as end of life for a long time now. Programming languages have very strong path dependence and it can be hard (or impossible) to put them on a different path.
I remember DHH being on the Django podcast and them asking about Rails’ async plans and DHH dodging that question so hard that I was reeling.
This is ridiculous… Ruby chose not to go the same route as Python with its async and sync function and instead has a unified system “a la” gevents but without monkey patching.
And Rails has supported async queries (the only part that really matters) for several years now.
Yeah this seems to be a fundamental property of software … early decisions live forever. Python is going through crazy contortions now to fix earlier architectural decisions. (We can call them “mistakes” in hindsight, but that assumes that you are able to predict the future, and should)
The most important component of evolution is death
It is easier to change existing software than to build new software, so software tends to live a long time. To a first approximation, software doesn’t die (compare this to the hardware in a computer, which is completely replaced every few years). At the same time, it is difficult to make major structural improvements to software once it has been shipped, so mistakes in early versions of the program often live forever. As a result, software tends to live too long: just good enough to discourage replacement, but slowly rotting away with more and more problems that are hard to fix. I wonder if the overall quality of computer software would improve if there were a way of forcing all software to be replaced after some period of time.
Not that I think Ruby should die … I think many great things have been built with it. Someone said “a benefit of choosing Ruby is that it attracts people who get things done”. And they do have a pretty good track record of successful businesses
So yeah it is too much to say “end of life”, but I’d say just “not for certain problems”, which is true of every language
Honestly I think a lot of software should die to prevent more misery from being brought into the world. Let’s please end of life PHP, salt the earth and hope people disperse and go for better alternatives.
I’ll not continue to harp on about Ruby but it’s not obvious to me that it should make the cut or will ever become a thing.
I feel like we’re missing out on what we could do with an actually dynamic desktop environment (think scwm or NeWS or Emacs or Smalltalk). GNOME’s experience trying to wedge stuff in via JS points to us needing this direction, but I don’t think anyone’s taken it seriously on Linux. Probably because it requires a runtime that can compile code, introspect itself, and handle isolating and managing components, and historically only some of the Lisps provided that.
I don’t have a hat for being one of the founders of Étoilé, but the project died a while ago. From my side, it was because the things that I wanted to do were really hard on existing hardware, so I went off to build the hardware that can do it. There were two important things I was trying to enable with Étoilé:
First, end-user programming at every level of the system. This meant building high-level languages that closely interoperated with low-level ones. I wanted people to be able to reuse existing things (no one should have to reimplement libavcodec / libavformat to be able to play video, for example) but in such a way that you can’t crash your program by using them wrong (or by having bugs in them). That meant that being able to sandbox them, without losing their performance, and being able to share data at the granularity of objects with them. If I started now, I might use WAsm to vaguely approximate what I want. With CHERI, I can actually build it properly.
Second, I wanted to get rid of applications as a user abstraction. I wanted a canvas, where you had different tools that could provide different objects and ways of interacting with objects. No word processor, drawing program, audio editing program, and so on. If you’ve used ClarisWorks, you’ve seen a glimpse of what I wanted: In ClarisWorks, every text field that you drew in the drawing program had the full power of the word processor. Every table you drew in the word processor had the full power of the spreadsheet. Being able to do that well required being able to compose bits of code from different sources (including random Internet people) and aggressively constrain what they can do, to the granularity of which objects that can touch, with good performance. Again, CHERI enables that.
With these, some other folks were working on things like the CoreObject data model, where you have a generic data model for tree-structured data, with extensible diffing and merging operations. This gave you unlimited branching persistent undo for free. It also gave you something that you could build live collaborative editing on top of very easily (which we did for some demos, wrapping the updates in XMPP messages and some people did in some iOS apps that used the framework), because you can extract a structured object diff and then send it to someone else who can then merge it, and you can do that every time there’s a change to the document.
The generic object model also allowed us to do some fun things for end-user programming because the view hierarchy was exposed as the same model, so you could use generic inspectors to inspect your current view hierarchy, and then inspect that, recursively. And do diffing and merging on views and build different views of the same data by using the same sort of inspectors that let you inspect the data.
To me, this more closely aligned with the Free Software ideology than an app-based model, because being able to modify, extend, and distribute programs was a core part of how you’d use the system. There wouldn’t be big monolithic and incomprehensible applications, there would be tiny tools (‘services’) that you could assemble to build rich environments. The apps-as-distinct-things model is closely tied to the economic model around selling off-the-shelf software and replicating that in a Free Software ecosystem makes no sense to me. Arcan has a similar philosophy.
So glad to get this reply to my mention of Étoilé. IMO, it’s one of those things that everybody should know about even if they can’t use it - like Smalltalk and Lisp (at least everyone here knows about those) and Forth and HyperCard.
Now, just personally, the part I cared about the most was the first part (your second paragraph) and I hope we get at least that part one day. Oh, plus I really liked the syntax and programming style you were using.
If you haven’t seen it, the STEPS project from VPRI is worth a look. They were one of our major sources of inspiration. A lot of what we were trying to do was a more pragmatic version of their system.
I have looked at some of the high level STEPS documents, but I wasn’t able to find an open source copy of the software that they wrote so that I could run it and play with it. Does this exist?
They did release some things, but I’m not sure where to find it. From my perspective, the ideas were all more interesting than the code. The code was driven by the Smalltalk mindset that it’s possible to rewrite everything in your favourite language and ignore the existence of other ecosystems, which is fundamentally limiting. As a tech demo, being able to write an entire OS and some apps in 20KLoC is nice, but it also lacks a load of features people actually want.
There’s a quote from one of the Sun founders that I haven’t been able to find a good attribution for:
Always remember that more smart people don’t work for you than do.
It’s a lesson that a lot of Smalltalk, Go, and Rust advocates could benefit from.
I mean, I’m glad that people are doing something, but none of this really changes the “Windows 95” model of interaction we’ve all been suffering from for 30 years.
Genuinely asking, what is the “Windows 95” model, and how exactly does it cause suffering, in opposition of other “models”? What is your definition of a “UI model” and which UIs fall under what models?
Does it use being written in Rust to achieve something Gnome, Cinnamon and pals don’t? Not to suggest necessity is the only reason some new software should exist, but what’s the driving factor behind a new DE? The fluff on the homepage just reads like fluff, and doesn’t say anything substantive about the project.
I think rust projects are meaningfully easier to contribute to than C, C++, or Python ones. Good build system, one auto formatter, package manager that solves transitive dependencies problem, reasonably conveniently typed language and reasonably advanced tooling go a long way towards being able to spend less than an hour between an initial git clone and a pull request.
I’d say, among popular languages, only go is comparable to (and perhaps even better than) Rust in this respect.
So, my baseline expectation is that they’ll be able to ship more and faster than KDE and Gnome, with less work.
Compare with how quickly people were able to ship fine-grained incremental compilation in rustc, because it was implemented in Rust.
I would like to second this. I would find it hard to overstate how much easier I find it to contribute to Rust projects, despite being a novice in the language. I’ve lost a lot of time in my life trying and failing to build projects so I could contribute to them.
That first build is a big hurdle that just is usually not present in Rust (or Go) projects. The usual case is git clone ...; cargo build. I find that’s a big convenience (and performance / memory safety / etc is just a nice bonus.)
package manager that solves transitive dependencies problem
Is it still the case when you have to rely on external dependencies? Not sure about this exact project, but I would assume that cross-language projects will need complex bootstrapping either way.
Yeah, external dependencies are unsolved. But in rust you generally can avoid them. Like, for example, I believe there’s a more-or-less complete port of harfbuzz to Rust? And that’s not someone just wanting to rewrite things for the sake of it, but rather than economic observation that Rust stuff is as fast and as reusable as C stuff, but is also memory safe.
Nope, but I still find it a lot easier in most cases with external dependencies.
E.g. The Helix editor requires one external dependency, gcc. That’s not a problem to me, and my experience with building Helix was still the practically painless cargo install.
Another e.g., the most complex Rust project I’ve built was Graphite, which is a Rust and Node project. I’ve already hit npm’s pain points before, so I didn’t hit any issues.
In both cases, Cargo doesn’t interact with the other package managers.
Seems questionable for a content organization to gate their content behind a framework as complex as React. Will the content work 20 years from now? Plain old HTML will. I wish the NYT had a lite version like CNN: https://lite.cnn.com/
This seems like a misunderstanding of where and how React is being used. I don’t know the internals of the Times CMS (although I’ve seen it demo’d) but it either stores content as some superset of plain HTML (XML, shortcodes, whatever) or as a series of JSON objects. Either way, the content has nothing specific to React about it and translates down to plain HTML and a format their print CMS can understand through the vagaries of the system. Special interactive stories are stored as one off pages on a static site. The content in those stories is frozen at publication time and then just never touched again after an initial period of updates. The Times also has lots of weird corners you aren’t thinking about like recipes, Wordles, and Wirecutter scores that are stored in their own bespoke ways. The React of this article is much more about the chrome of the page than its content. The focus of the article was how to deal with interactive widgets being injected into the page without React (which controls the page chrome) noticing and complaining. The widgets themselves could be written in any JS framework and many of them don’t necessarily follow best practices for 2024.
Yeah, if you had a React page and you wanted to come back to it and edit it after leaving it alone for a couple of years, it would be very difficult because the source would have rotted and become unbuildable in the meanwhile. But that’s not something the Times worries about because pages are either finished projects in which case they are frozen and never go back or living projects in which case they never go stale.
My feeling is that this is a rather technical solution to fix people’s bad habits.
In other languages, the fix would have been a linter that prevents you from creating jobs that run a loop, unless the only thing done within that loop was to create child jobs.
I don’t think of this as covering up bad habits so much as a new tool to help with tasks that need to be accomplished serially. A long running data migration or backfill, for example.
I have had to do this many times and while it’s not a huge deal to structure jobs to avoid the dreaded long running job, it’s nice to have the pattern ensconced in the library and not require rebuilding at every new app and/or educating engineers on how to do it.
Next step, presumably, is to take a load of money from Google to make them the default search engine and then spin up a for-profit subsidiary and then spend all of the money on things that are totally unrelated to the core mission?
Nah, that’s so last decade. Instead, we’re going with a strict “no default search deals, crypto tokens, or other forms of user monetization, ever” policy :)
Browser diversity means a lot to me. I donated $100 for the Firefox 1.0 ad in 2004, and I’m happy to donate $100 to Ladybird now, 20 years later. Good luck.
Seeing this post has unironically made me more optimistic about the future today. Obviously this is a huge undertaking but it’s sooo necessary, especially with mozilla and firefox drifting ever farther away from the interests of their users and generally falling apart.
I will be very curious to see what stance Ladybird takes on DRM in the future, jwz convincingly argues that implementing DRM was mozilla’s original sin.
The analysis isn’t particularly apt: Before EME, DRM video in browsers (including Firefox) was played using multiple proprietary plug-ins that were unconstrained in what they could do. After EME, depending on platform, DRM video is played either using a smaller and constrained proprietary plug-in or by underlying OS/device facilities. jwz is trying to make readers believe that going from multiple large unrestricted proprietary plug-ins to a smaller and more constrained one was the worst thing.
(Disclosure: I was a Mozilla rep in the HTML Working Group at the time.)
I’m not sure I find that particularly convincing. Isn’t the only argument “somebody else would have done it as a plug-in”?
Anyway, I am excited about ladybird and curious to see how they handle it. Maybe you just build a web browser without DRM and tell people to use something else for Netflix? Maybe you do add some kind of plug-in thing…? I have no idea how widevine actually works, given how powerful sidechannel attacks are I am just surprised that it isn’t entirely broken but maybe nobody cares enough.
Re widevine, it has been broken many times. It’s very simple: the CDM (content decryption module) contains a private key, the server checks that the public key you’re using is in a list maintained by Google, and then it uses the key to encrypt the content for you. This means widevine becomes a game of putting the private key in as hard a place to reach as possible and rotating it as fast as possible when it gets found.
The key is often hidden in a TPM or some other piece of hardware, but most places where Widevine is usable don’t have such a thing available, so it’s just stuck in the libwidevine.so that ships in every Chrome and Firefox. You might wonder “where could you possibly hide the key in userspace?” and the answer is “whitebox cryptography” which means creating a custom encryption function with the key compiled in and running a bunch of mathematical transformations to make it hard to get the key out of it. This is somewhat ineffective though - I believe on three separate occasions I’ve seen people talk publicly about extracting the key with a few math tricks, so I’m sure it happens way more in private.
The article argues at the same time that it needs to be fought and that it’s ineffective at preventing piracy which seems odd.
There are also arguments that “other protect content would follow” but afaik that hasn’t actually happened. For example the recentish WEI proposal was shot down rather quickly.
The article argues at the same time that it needs to be fought and that it’s ineffective at preventing piracy which seems odd.
Not really: actually a lot of things don’t work at their stated function, and are still a problem. The problem with DRM isn’t that it stops piracy, it’s that it stops a bunch of legitimate use of your computer, and requires locking down your personal device to a degree that you don’t know what’s going on within it, can’t change it, and can’t even rely on others to reverse-engineer it to find out what is happening. Whether it works to fight piracy is irrelevant to those concerns. It just makes it extra-frustrating to try and argue against when it’s clear that it’s a dumb idea that doesn’t even work at what it aims to do.
(The reason why it was felt it doesn’t decrease piracy, btw, is that you only need one copy and a distribution mechanism to provide to everyone the pirated copy that EME is supposed to prevent.)
Really, everybody I spoke to conceded that it was less an effective anti-piracy mechanism as a contractual requirement by corporate rightsholders so far up the distribution chain that it was impossible to renegotiate. E.g. Netflix had these contracts to stream, but in order to comply with the contract, they needed some sort of DRM.
Any DRM would do: it’s not its technical capabilities that would be important, it would be primarily the legal protections and liability it provided for the rightsholders.
I think the fact that DRM’s protection is primarily legal (and IP-based) is one that downstream developers often struggle with. Like, you potentially get the legal protection the lawyers want with a XORed stream, but most technologists are going to try and build a /really good/ DRM system, even when that involves re-engineering the entire general purpose PC to lock down certain content streams. So you get all of this investment and effort to make copying locally difficult, but never impossible, creating all of these problems for real customers and adding complexity to the base platform. But pirates are just going to pull stuff out of the analog hole and then distribute it for free or cheaply anyway!
There are also arguments that “other protect content would follow” but afaik that hasn’t actually happened. For example the recentish WEI proposal was shot down rather quickly.
WEI was shot down, but it was also proposed, which was the problem with EME: before EME there was a dependable line the community could draw at the W3C, so that if proposals go this far, even with hardcore pressure from corporations or rightsholders, the W3C could give a principled reason why they would not except them. Now there’s precedent to override that principle. It’s much better to have a defensible line – or preferably multiple defensible lines – than to keep on having to fight the same fight, having fallen back from a previously hard line.
I think in the modern era we’re going to see an interesting shift in the conversation. For the past few years, piracy has been (perceptually) dropping online because the average person doesn’t do it – the convenience of streaming overrode any desire to get an unauthorised version. Now that streaming is increasing in price, and also fragmenting, the incentives will increase to defect, and return to sharing infringing content. I think we’re going to see a harder moral sanction against doing that, both because the zeitgeist is more in favor of shame, and because the battles over things like the DMCA are much more remote. The idea that it’s illegal for me to even describe here how to get around EME may well seem less outragious to a generation that has grown up with those kind of legal restrictions, and pervasive DRM.
Also, I’ll note that the increasing call for non-machine-readable content is going to make strange bedfellows of AI skeptics and copyright maximalists. What better way to defend your material from marauding AI bots than to wrap it up with DRM? Snakeoil salesmen don’t care who their audience is, they just change their pitch.
I still think that – like encryption – most involved technologists understand the contradictions that lie at the heart of regulating or implementing DRM. But every year we live in a DRM, locked-down, world of non-general-purpose computers, the harder it is to paint this as an absurdity.
Please fix the donation system. It refused to accept recurring donation from my MasterCard, refused one-time donation and finally started to just display “Forbidden”.
EDIT: My bank has just informed me that Donorbox are fraudulent and people have repeatedly reported being charged without their consent so they have blocked the transaction. Please use a different provider. This resulted in my card being blocked and I am being issued a new one.
I see the website says you will soon have 7 full-time software engineers on the project… that’s a lot of money! Who is the money mostly from? Why are they donating it? How do you know it will continue?
This is not to say I don’t believe the project can succeed - I just want to understand who believes in it enough to put this much money behind it, and why. Most entities don’t want to put up this kind of money without some ROI.
Looks like most of the money comes from one of the two founders/board members. While their website also lists sponsors, it does not yet seem like enough to keep things going. I really like what they set out to do here, but am still curious to know the plan for when the $1M is spent, in particular, since they say they want to grow the team further.
spend all of the money on things that are totally unrelated to the core mission?
To be fair, I think we are better off with Rust than without it, and I’m glad that Mozilla funded it into existence. I’m sure that Mozilla did a lot of less convincing stuff, but they probably also did a lot of nice stuff that is less well-known but still had positive impact for the web. (For example, I suppose they played an important role in setting up the Javascript specification process in a reasonable way, asm.js is also not bad, presumably they did weight in to support open codecs and open standards, etc.)
You may know better than I do, but the vibe that I got from Mozilla for a long while was that it was a nice place to work for people who are passionate about open source and the web, and people had agency to work on things that they believed actually matters. This seems like a good way to go about things, which I think they deserve credit for. Sure, there were a few debatable decisions thrown in the mix as well, mostly dumb and values-contradicting ideas that were supposed to help make the project financially sustainable without Google backing, but I would still count Mozilla as a net positive.
(In comparison I find it harder to find excuses for Ubuntu that, while overall delivering a solid product that people want to use, seems to have a knack for make repulsive decisions all the time.)
To be fair, I think we are better off with Rust than without it
As I understand Mozilla’s involvement, Rust was largely a spin-off of Servo: The latter was a project to build a memory-safe browser engine, the former was the language built to realise that goal. Mozilla dropped funding for both. Rust managed to survive but there was quite a disruptive period when a load of key Rust contributors suddenly stopped being funded by Mozilla (2020), including one member of the Rust core team.
Servo absolutely was something that was core to Mozilla’s mission of promoting an open web. I’d love to use a web browser where the entire rendering pipeline was implemented in a memory-safe language. Mozilla fired all of the Servo team in their 2020 layoffs. They found a new source of funding last year but there was a lot of disruption.
The entire announcement makes me happy, I’m just sad about the reason it makes me happy: That this was exactly what the original Mozilla Foundation announcement looked like 21 years ago (almost to the day) and the open web really needs a solid competitor to Chrome (just as, back then, it needed a solid competitor to IE). Mozilla consistently compromised on principles and directed funding to projects that were unrelated to their core vision (but not quite enough funding for any of those projects to actually succeed).
Yeah, I remember it too (and I continue to sigh whenever I need to e.g. remove Pocket from the toolbar of a new Firefox install and whatever else). I suppose Ladybird has the advantage of foresight of such issues that Mozilla didn’t.
If they don’t have a sustainability plan today, they will adjust their ethics when one presents itself. Happens every time. Just wait until they take VC.
I still find testify incredibly useful and assert that Go needs an assertion package.
I also totally agree that it is the height of folly to re-invent tools from other languages (e.g. RSpec and Cucumber) in Go. The aside about Gomega and Ginko is dead on.
That looks nice enough, and the .Equal uses generics, which is good.
So a basic not-an-err check would be qt.Assert(t, qt.IsNil(err)), compared to require.NoError(t, err).
(most of the check functions are func abc(got, want T) Checker, which is, sure, arbitrary, bot also exactly the other way around from testify, which tend to do (want, got))
I use quicktest for work, and it does the job.
There are some warts though:
Its pretty print (that happens on failed assertions) is a bit naive and will fill your screen with garbage if there’s at lot of stuff in the things you’re asserting on (for example a big array of bytes). If it’s big enough your test will hang for a while without printing anything (this happens fairly reliably if there is a context or a protobuf struct in there).
Writing custom assertions is kinda weird.
Goland (the IDE) does not recognize quicktest subtests so one cannot run the subtest by clicking on it. Not really quicktest’ fault but still to be considered.
My “favourite”* thing related to Nginx is Avoiding the Top 10 NGINX Configuration Mistakes. To me this entire post can be summarised as “our default are bad and we should feel bad, but we’ll make you feel bad instead”.
Nginx has so many footguns that I just can’t recommend it for anything other than very simple or hobby setups anymore. It was great at being an Apache httpd alternative 20 years ago, but it’s really showing its age now.
* Meaning, wow this post should not have to exist.
No but really, Mistake 6 is “if you use an if statement in the wrong kind of scope, you can get memory corruption”.
That’s right, nginx.conf is a memory unsafe language. And they could 100% check for this one at config-load time, it’s entirely a static (syntactic, even) property. But they don’t, because there is no safety culture there.
Oh damn, they took that down? That was one of my favorite things on the internet!
Anything else may possibly cause unpredictable behaviour, including potential SIGSEGV.
It is important to note that the behaviour of if is not inconsistent, given two identical requests it will not randomly fail on one and work on the other, with proper testing and understanding ifs ‘’’can’’’ be used.
It depends on the use case. Caddy is great for simple setups and gets you TLS for free. You can’t do that with Nginx. (Caddy is great for more complex setups too, but I mention simple here because you can run it with almost no configuration and have working system with TLS.)
Traefik is next on my list. Great for containerised environments as it can do auto discovery. It has a lot of tricks up its sleeve, and also has automatic TLS support. More configuration needed than Caddy.
For complex requirements, such as that described by the OP, I’d suggest Envoy. It’s a complex beast, but very powerful. At work I’m starting to replace our Nginx-based proxies with Envoy, and am writing a custom control plane for it.
For large or complex deployments I’m generally in favour of having the centralised control plane approach so that all your proxies get configured consistently. With Nginx I’m tired of having masses of templated configuration files and getting them in to Kubernetes pods.
Nginx has so many footguns that I just can’t recommend it for anything other than very simple or hobby setups anymore. It was great at being an Apache httpd alternative 20 years ago, but it’s really showing its age now.
I suppose this is hyperbole but if my memory is not failing me, nginx was just a blip in 2006-2007 era and was all the rage up until 2014-15, so I’d say it was absolutely great 10 years ago, I can only not comment about the state after that.
That doesn’t mean it wasn’t perfect or had no footguns, but you’re making it sound like it was a liability. It was not.
Snap is a tragedy. I hate it. I will likely switch to Fedora due to it.
Fedora has Flatpak, which is just as bad. Back to the drawing board with all of this, whatever they were trying to accomplish it’s not worth the downsides.
It’s really not as bad. All the usual snap complaints don’t apply here: Basic system applications are not flatpaks. You can uninstall flatpak completely. Flatpak doesn’t auto-upgrade. You can downgrade flatpak apps to older versions if you want, or pin them to a specific one. Fedora doesn’t carry any “rpm that actually just installs a flatpak” packages. You can have multiple custom flatpak repository sources.
Did I miss some snap issue that’s present in flatpak and not optional?
Did I miss some snap issue that’s present in flatpak and not optional?
Snaps are compressed on disk and usually need to uncompress to run which can cause significant delays in statup time. Flatpaks aren’t stored compressed and thus there is no startup delay like snaps.
This is news to me! Totally a shot in the dark and a bit naive, but couldn’t something like zram help here? I guess it’s still the decompression (and that apps can easily fill that space).
The issue is that installed snaps are still compressed disk images, with compression settings determined by the publisher (hence Canonical working with Mozilla to improve the start-up time of the Firefox snap) and very limited, publisher-dependent support for deduplication via the ability to define a stack of disk images to build an overlay filesystem from, and which must be decompressed every time you load from them while Flatpaks are installed using a “git for your filesystem” framework (OSTree) which decompresses (or hands off to your filesystem’s innate compression support which you can tune) at install time and deduplicates any hash-identical files across your entire installed base of applications.
It’s a fundamental shortcoming in Snap’s design.
Snap just generally does a lot more lumping of concerns together into a single system than Flatpak, where they try to focus on well-factored components. (eg. Snap is trying to be both a Flatpak competitor and a system for immutable distros. Flatpak is just an integrating wrapper on top of the OSTree system that immutable distros like Fedora Silverblue also use for system packages via rpm-ostree. Likewise, Flatpak’s sandboxing is a separate tool you can use independently named bubblewrap.)
It’s that an issue? If you want to save space, you can mark the whole flatpak store for compression (if you’re using btrfs or zfs) and you’ll gain both quick startup and disk saving
I currently run Pop!_OS, a Debian-based distro that ships with flatpak. Their app store includes both flatpaks and debs, and doesn’t make a clear distinction between them. I always curse them when I click the wrong one.
My main gripes with Flatpak are:
The packages are substantially larger than debs. Analogous to containers rather than packages.
The startup time is noticeable. I always have a “oh shit, what went wrong” reaction when I launch a flatpak, before it finally launches.
KDE itself is on the verge of dropping support for X11. The latest release’s release notes made it very clear that new feature development is happening in the Wayland version now.
it is, but that doesn’t help me when i’m under wayland and a feature I rely on doesn’t work because Wayland plain does not support it (last one in date : moving the mouse cursor to an arbitrary position from the app to implement infinite scrolling - this works on macOS, Windows and X11 but Wayland just doesn’t allow arbitrary apps to do this at all which makes a basic GUI operation in any pro audio/video software a PITA instead)
I’m an X11 holdout but I’m wondering if I’m going to have to actively engage with fixing Wayland at some point (or if some new thing might come along and displace it before we’re done reimplementing X…)
In the last couple of months, we’ve been seeing work on improving rootful XWayland so, if the necessary user-empowering features aren’t ready by the time LTS distros drop bare-metal X11, it should be possible to do something like running a kiosk-oriented Wayland compositor which runs rootful XWayland as its single application, then dig up something like kwin_wayland’s option to run with an X11 backend, but with support for rootless operation.
If you’re writing software against Qt directly, or KDE’s libraries, it usually is. Kwin itself is an implementation of a display manager, so obviously has to care quite a bit about the underlying protocol.
What I read is that they want to continue supporting X11 for the forseeable future. And I’m glad they do. Should that change, Linux desktop is in serious trouble.
I don’t need new features. But I need my computer to work. I need to be able to record my screen, share my screen, setup global hotkeys, simulate inputs… And I need to do that in a standard DE-agnostic way!
So X11 it is! And if one day KDE drops X11 support, I’ll be forced to drop KDE.
I’ll gladly let more enthusiastic people beta-test this one for me. Maybe one day Wayland will be usable (and no, last I checked was less than a month ago! I know it doesn’t work for me).
I don’t want to sadden you, but plenty of DE’s support for X11 is already minimal, even bugs get low priority in fixing. I know this is the case for Gnome, for example.
Interesting, my experience with Flatpak on Fedora is that applications open in about 2 seconds. This is specifically on Aurora, where just about every application is Flatpak’d. Applications taking 20 seconds longer to start up under Flatpak than natively sounds like a Flatpak bug, not expected behavior.
For an article that references XKCD, let me provide another: https://xkcd.com/927/
Gotta be honest, I don’t see the point:
I find it less comfortable. Current SemVer is strictly objective - if breaking, then major. Else: if affects externally-visible behaviour, minor; else patch. BreakVer requires me to make a subjective judgement about what constitutes a major breakage - which requires knowing about all my customers - which is, practically, impossible. Plus, depending on my confidence/arrogance/move-fast-and-break-things-iness, I’ll make classifications that do not match up with customers’ expectations.
That is currently true. You’ve just shifted the boundary of specificity - instead of subdividing non-breaking changes, you’re sub-dividing breaking changes. In practice, either:
…do they? Why? Genuine question.
My descriptivist take is that SemVer isn’t what the spec says, but what the tooling does with it, and how people are affected by that.
If you bump the first number, people won’t update quickly, and this may be a good thing or a bad thing, depending on what your goal is.
If you bump the other numbers you may or may not upset a lot of people and get complaints.
So you weigh the risk of getting an avalanche of complaints vs how inconvenient it would be for you if users didn’t update your software quickly, and maybe also complained it doesn’t update automatically often enough.
Is it?
As the article states, “There’s an old joke that every change is breaking for someone.” I think that the joke is true to a large extent.
Yes, it is. It doesn’t matter whether the change is breaking “for someone” - it matters that the change is breaking to a public contract. If you take a dependency on behaviour that isn’t an explicit part of the an established contract, you have no right to expect warning when it changes.
I find it jarring when a piece of software goes from version 16.213.26 to 17.0.0 just because the developers removed spacebar heating.
“breaking” is not objective. Remember Hyrum’s law. Someone is going to need you to re-enable spacebar heating if you pretend it is.
Breaking in semvar is objective. It’s defined by the spec as a change in the behavior of your public interface. If you haven’t documented your public interface, you aren’t following semvar.
While this may be strictly true, it also implies that almost no one is actually following semver, which doesn’t seem like a very productive framing to me.
Huh? When the semvar spec says “public API”, I imagine some docs you can read that list all of the functions in a package along with their type signatures and a description of their behavior. Most of the packages you use have this, no?
That’s the package’s public interface. If the package changes one of those type signatures, that’s a breaking change. If it introduces a new function, that’s not breaking. If it makes a change that violates the documentation for one of its functions, that’s a breaking change. If it makes a change to the behavior of a function that’s consistent with that function’s docs… well either that’s not a breaking change, or as is common the function was inadequately documented.
This all seems fairly unambiguous to me, excepting changes to the behavior of poorly documented functions. Am I missing something?
The example I’ve gone round and round a bunch of times with people on is: Go 1.13 introduced the ability to use underscores as grouping separators in integer literals, like
1_000_000instead of1000000.This also changed the behavior of Go’s integer-parsing functions. For example,
strconv.ParseInt()suddenly started accepting and parsing inputs with underscore characters rather than returning an error. And the Go team seem to have been aware that there were people whose code was broken by this change, which would be a problem for Go’s claim that there will never be breaking changes ever, for any reason.Generally people have argued with me that although
ParseInt()was a public function, it was somehow underspecified or ambiguously specified prior to Go 1.13 and therefore it was acceptable to clarify its behavior in Go 1.13 by suddenly changing the inputs it accepted. But this just points to the real purpose of SemVer: it’s about protecting the developer of the code from the user of the code, by giving the developer endless subjective loopholes and ways to say “sure, that change broke your code, but it’s still not technically a breaking change”. For example, any function which does not specify up-front the entire set of potential inputs it will accept and the results it will return for them is subject to theParseInt()style of “oops, we underspecified it” loophole.Ah, I get it. There are three things the Go docs for ParseInt() could say:
Reading the actual docs, I’d frankly put them in the first case: they state what the accepted integer syntax is, and give examples, and all of this makes it rather clear that underscores aren’t part of the integer syntax, any more than “thirty seven” would be.
But even if the docs weren’t clear, you don’t get to say “LOL no change is breaking change because I forgot to write docs”. That just means you’ve entered a gray area, and you should be cautious about what counts as a breaking change and your users should be cautious about not relying on too much. It should be a “meet in the middle” sort of a thing, not a “how C++ compiler writers interpret undefined behavior” sort of a thing.
tldr; I’m sorry that people are treating “incompletely documented” the same as “explicitly unspecified”, those are very different things.
Isn’t accepting “1_000” in Go source also a breaking change by the same reasoning as it would be for ParseInt? Input that used to result in an error no longer does.
Maybe in some technical sense, but people rely on both positive&negative behavior of ParseInt() (e.g. rejecting invalid user input), but generally only rely on positive Golang behavior. If “this program started to compile when it used to be an error” was considered a breaking change, every change in language behavior would be breaking.
Why do you keep calling it “semvar”? It’s “semver”, semantic versioning.
Just a typo. Past the edit window so I can’t fix it now…
What I meant was that in the real world, it’s very common for an API to be underdocumented, with the result that it’s not well-defined whether a given change breaks the API or not. Like, you can look at certain changes and say “this really seems like it breaks the API,” but the API was defined vaguely enough that it’s impossible to make any judgment like that.
You say “…excepting changes to the behavior of poorly documented functions,” but I think that’s actually a very large category, in practice :-) Obviously there are some libraries and frameworks that take defining their APIs very seriously, but I would guess that the set of libraries that use SemVer is an order of magnitude larger than the set of ones that are strict about it in this way.
Yeah, that all makes sense. I’d argue that if it’s hard to make that judgement call, the library should be conservative and bump the major version number.
Is there a versioning system you think does better in the presence of a poorly defined API?
I don’t know of one, and I suspect that carefully defining the API is a prerequisite for any versioning system to be able to give you the kind of guarantees we want.
Hyrum’s Law describes a common pathological condition of dependency relationships between software modules, it doesn’t define a de facto baseline/standard guarantee or expectation of compatibility.
That person is welcome to be upset and to ask for consideration, but they are by no means entitled to it. SemVer is about breaking the explicit, published contract of software, not about breaking any hypothetical consumer. If you take a dependency on undefined behaviour, you have no rights to complain when it changes, nor any justification to expect warning when it does.
I’m concerned that Bluesky has taken money from VCs, including Blockchain Capital. The site is still in the honeymoon phase, but they will have to pay 10x of that money back.
This is Twitter all over again, including risk of a hostile takeover. I don’t think they’re stupid enough to just let the allegedly-decentralized protocol to take away their control when billions are at stake. They will keep users captive if they have to.
Hypothetically, if BlueSky turned evil, they could:
This would give them more or less total control. Other people could start new BlueSky clones, but they wouldn’t have the same network.
Is this a real risk? I’m not sure. I do know it’s better than Twitter or Threads which are already monolithic. Mastodon is great but I haven’t been able to get many non-nerds to switch over.
Hypothetically, the admins of a handful of the biggest Mastodon instances, or even just the biggest one, could turn evil and defederate from huge swathes of the network, fork and build in features that don’t allow third-party clients to connect, require login with a local account to view, etc. etc.
Other people could start clones, of course, but they wouldn’t have the same network.
(also, amusingly, the atproto PDS+DID concept actually enables a form of account portability far above and beyond what Mastodon/ActivityPub allow, but nobody ever seems to want to talk about that…)
The two situations are not comparable. If mastodon.social disappeared or defederated the rest of the Mastodon (and AP) ecosystem would continue functioning just fine. The userbase is reasonably well distributed. For example in my personal feed only about 15% of the toots are from mastodon.social and in the 250 most recent toots I see 85 different instances.
This is not at all the case for Bluesky today. If bsky.network went away the rest of the network (if you could call it that at that point) would be completely dead in the water.
While I generally agree with your point (my timelines on both accounts probably look similar) just by posting here we’ve probably disqualified ourselves from the mainstream ;) I agree with the post you replied to in a way that joe random (not a software developer) who came from twitter will probably on one of the big instances.
For what its worth I did the sampling on the account where I follow my non-tech interests. A lot of people ended up on smaller instances dedicated to a topic or geographical area.
While it’s sometimes possible to get code at scale without paying – via open source – it’s never possible to get servers and bandwidth at scale without someone dumping in a lot of money. Which means there is a threshold past which anything that connects more than a certain number of people must receive serious cash to remain in operation. Wikipedia tries to do it on the donation model, Mastodon is making a go at that as well, but it’s unclear if there are enough people willing to kick in enough money to multiple different things to keep them all running. I suspect Mastodon (the biggest and most “central” instances in the network) will not be able to maintain their present scale through, say, an economic downturn in the US.
So there is no such thing as a network which truly connects all the people you’d want to see connected and which does not have to somehow figure out how to get the money to keep the lights on. Bluesky seems to be proactively thinking about how they can make money and deal with the problem, which to me is better than the “pray for donations” approach of the Fediverse.
Your point is valid, though a notable difference with the fediverse is the barrier to entry is quite low - server load starts from zero and scales up more or less proportionally to following/follower activity, such that smallish but fully-functional instances can be funded out of the hobby money of the middle+ classes of the world. If they’re not sysadmins they can give that money to masto.host or another vendor and the outcome is the same. This sort of decentralisation carries its own risks (see earlier discussion about dealing with servers dying spontaneously) but as a broader ecosystem it’s also profoundly resilient.
The problem with this approach is the knowledge and effort and time investment required to maintain one’s own Mastodon instance, or an instance for one’s personal social circle. The average person simply is never going to self-host a personal social media node, and even highly technical and highly motivated people often talk about regretting running their own personal single-account Mastodon instances.
I think Mastodon needs a better server implementation, one that is very low-maintenance and cheap to run. The official server has many moving parts, and the protocol de-facto needs an image cache that can get expensive to host. This is solvable.
Right! I’ve been eyeing off GoToSocial but haven’t had a chance to play with it yet. They’re thinking seriously about how to do DB imports from Mastodon, which will be really cool if they can pull it off: https://github.com/superseriousbusiness/gotosocial/issues/128
Worst case one moves off again. That’s a problem for a future date.
That’s true, but I’ve been hooked on Twitter quite heavily (I’ve been an early adopter), and invested in having a presence there. The Truth Social switcheroo has been painful for me, so now I’d rather have a smaller network than risk falling into the same trap again.
Relevant blog post from Bluesky. I’d like to think VCs investing into a PBC with an open source product would treat this differently than Twitter, but only time will tell.
OpenAI was a “non profit” until it wasn’t.
OpenAI never open sourced their code, so Bluesky is a little bit different. It sill has risks but the level of risk is quite a bit lower than OpenAI was.
OpenAI open sourced a lot and of course made their research public before GPT3 (whose architecture didn’t change much[1]). I understand the comparison, but notably OpenAI planned to do this pseudo-non-profit crap from the start. Bluesky in comparison seems to be “more open”. If Bluesky turned evil, then the protocols and software will exist beyond their official servers, which cannot be said for ChatGPT.
[1]: not that we actually know that for a fact since their reports are getting ever more secretive. I forget exactly how open that GPT3 paper was, but regardless the industry already understood how to build LLMs at that point.
Would it be reasonable for GitHub to make this default behavior?
No, but it should be optional. Gitlab supports this: https://docs.gitlab.com/ee/ci/pipelines/settings.html#auto-cancel-redundant-pipelines.
There are some cases where I do want different commits on the same branch/pull request to run all actions. For example, if I am updating workflow dependencies for a workflow that normally runs on merges to main, I will temporarily make them run on my branch to test/validate. I make two commits—one with the updates, and another that runs the workflow on my branch. Once I have validate the change, I remove the last commit.
It is optional, using exactly the process described in the linked article!
I love this opinionated POSIX standard. Instead of baking in a bunch of hacks to support filenames with newlines, they just said “don’t do that, not supported going forward”. That’s a change I can get behind.
what will happen to existing files tho? i have some, currenty all programs support those without any hacks, except one (
ls). will i be unable to even rename or delete them if i mount my storage on a new posix 2024 system?The article says:
thank you, i need to read more care fully (either i missed “create” or assumed create a filename means create a parsed path object from a string)
I’m curious how this plays with locales. The most common places I’ve seen this issue are where the file is created with something like a Big5 locale and then viewed in a C locale.
Trail byte on Big5 is 0x40 or higher, so not line feed.
It would really make POSIX ready for 2024 to drop non-UTF-8 locales, though. (Which probably won’t happen as long as someone finds it important to be able to claim a Big5 AIX system POSIX-compliant.)
I see the issue as one of compatibility:
At some point I need to write an actual article about this, but I’ve recently been thinking that “blocking” APIs in general are a design mistake: everything is fundamentally concurrent, but OSes use blocking APIs to hide this concurrency, which tends to inevitably break, and mixes concurrency with parallelism. Even something as simple as “spawn a process, collecting both its stdout and stderr” requires proper concurrency. See how Rust’s
Command::outputcan’t be expressed using std-exposed APIs.Which is the smaller problem! The bigger problem is that we don’t actually know how to write concurrent programs. This is an unsolved area in language design. And the systems reason for why the problem is unsolved is, thanks to the existence of blocking APIs, languages are allowed to shirk here, moving what is fundamentally a language-design issue to the OS-design.
IO-uring (and previously, hilariously, JavaScript) models the problem of concurrency more directly, without introducing a midlayer. Hopefully it will create enough of evolutionary pressure for language design to get the problem solved finally.
There was an interesting period about 25 years ago, when the web was growing fast and really stressing the capabilities of the hardware and operating systems of the time. This is when the c10k problem was identified. The performance pressure led to a bunch of interesting operating systems research.
I’m pretty sure I read a paper around that time which described a kernel/userland API with a design roughly along the lines of io_uring — though its point of comparison was hardware interfaces with command queues and DMA buffers. I had a very vague memory of Vivek Pai’s IO-Lite but that has a different shape so there must be another paper I’m failing to remember.
I wondered if a fully asynchronous kernel API could solve the M:N threading problems that were happening around the same time. But that whole area of research dried up, as the problems were dealt with by
sendfile(), 1:1 threading, and computers getting faster. Boring!I do not actually see a clear specification of what the problem is in that article, though.
It was really hard to handle 10,000 concurrent connections from one server in 1999. The challenge was to make it easier, to make the kind of performance engineering that went into cdrom.com available to a lot more systems.
IIRC, the fundamental challenge is handling 10K long-running connections that are not very active. If each of those connections is actually producing significant load on the hardware, either CPU load or I/O load, then the hardware can’t handle 10K connections anyway, because we have neither 10K cores nor 10K independent I/O channels.
You’re probably thinking of Matt Welsh’s SEDA paper.
https://search.worldcat.org/title/892830129
I found a copy of the SEDA SOSP paper which is a bit shorter than Welsh’s PhD thesis :-) It seems to be about getting more concurrency out of Java, not about changing the userland/kernel API.
It’s a compelling argument, but I think in some ways async APIs end up constraining the design space of the framework/OS/whatever offering it. For example, QNX’s main message passing primitives are blocking so that, on the fast path, context is switched to the callee without a pass through the scheduler.
More broadly, this paper argues that it’s not possible to make a fast and DoS-resistant async message passing system: https://www.researchgate.net/publication/4015956_Vulnerabilities_in_synchronous_IPC_designs
You absolutely hit the nail on the head. Given how absolutely fundamental asynchronous computation is to how computers work, it’s surprisingly underexplored.
I think this is partly due to a decline of groundbreaking OS research, and partly because there are barely - if any - modern languages purpose built for doing deep, systems research. The problems become apparent almost instantly. At the latest when you’re a few hundred lines into your first kernel, you’ll be presented with DMA interfaces for all kinds of peripherals. What now?
This is not the kind of problem that most language designers are facing. Most PLs are made to solve application-level problems, instead of having a principled, hardware-first approach to language design. I think Rust with embassy-rs has been solving some of these challenges in a pretty nice way, but it shouldn’t end there.
I agree on the lack of OS research. Mothy gave a keynote at SOSP a few years ago pointing out that twenty years ago there were multiple new kernel papers each year at OSDI / SOSP whereas now we average less than one. I’m slightly biased, because the last paper I submitted with a new OS design was rejected on the ground that we didn’t do a performance comparison to an OS that required ten times as much hardware and that we were possibly vulnerable to an attack that, by construction, cannot happen on our system. Ho hum.
I disagree on language research though. There’s a lot of interesting work on asynchrony. Most PL research takes at least ten years to make it into mainstream programming languages. Verona’s behaviour-oriented concurrency model is designed to work directly with things like DMA and we’ve ported it to C++, Rust, and Python. It works best with a memory management model that puts object graphs in regions and allows them to be atomically transferred, which depends on a type system that can do viewpoint adaptation. We’re not the only people working on things like this and I expect mainstream languages in the 2030s to have a lot of these concepts.
Timothy Roscoe, “it’s time for operating systems to rediscover hardware”
Which kicks off with the observation that only 6% of the papers at the USENIX OSDI conference were about operating system design and implementation.
That’s brutal. Too brutal in my opinion to only come from spurious rejections. From the outside, I would suspect the main drive is that nobody makes kernels any more: apart from specific niches like real time or formal verification, the world has basically been taken over by NT, Linux, and XNU. And a major reason behind such concentration is that every kernel needs to have dedicated drivers for the insane diversity of hardware we use — except NT, hardware vendors being business savvy enough to write the driver for us. (See The 30 Million Lines Problem.)
Those three kernels are big. Making any addition or change to them is likely a significant endeavour. We likely need the ability to quickly write practical kernels from scratch if research is to take off again. But no one can possibly write a practical kernel if they have to, let’s be kind, port all the driver code from Linux. So the first step has for hardware vendors to design interfaces for humans, and then give us the manual. But given how suicidal this would be business wise (who would ever buy hardware that doesn’t come with a Windows driver out of the box?), we probably need regulators to step in: say the US, or the entirety of the EU, ban the sale of any hardware for which the same company distributes software — or at the very least, proprietary software. With such a setup, I would expect hardware vendors to quickly converge on relatively uniform hardware interfaces that aren’t stupidly complex.
But then there’s the other side of the fence: user space. Since the ossification of OSes we came do depend on a huge stack of software that would take significant effort to port anywhere not POSIX-ish. And that’s for the stuff we have the source code of…
I don’t really think that’s true. For a research project, you need a few drivers and most people just pick up the NetBSD ones (the RUMP kernel work makes this trivial). There’s a big unexplored space for both big and small operating systems. With a very small team, we were able to write an RTOS that has a solid FreeRTOS compat layer and can run most existing embedded software (source compatible) and we were designing and building the core that ran it as well. No one has really built an OS for cloud computing, but it could remove a lot of stuff that desktop operating systems need (no IPC or storage layers, since those will both sit on top of a network).
fwiw I think existing unikernels fit that bill (e.g. unikraft, nanos). They mostly target virtualized hardware like KVM, and seem pretty focussed on the cloud use-case (fast boot, small images, and efficiency). Being in KVM-land certainly removes a lot of implementation complexity, since bringup and platform init is mostly done.
I disagree. I think unikernels are applications for cloud computing. They’re hampered by the fact that the OS that they run on (Xen, Hyper-V, Linux) is not designed for the cloud. They have network devices, for example, but a good cloud OS would make it easy to take advantage of hardware TLS offload and terminate connections directly in a unikernel.
Unikernels have had almost no commercial adoption precisely because there isn’t a good OS to run them on. The benefits of unikernels are incompatible with the billing models of cloud systems. A unikernel invocation may consume a few MiBs of RAM and a few CPU seconds (or hundreds of CPU milliseconds). The operating systems that run them are not set up to handle this at all. They are intended to account for CPU usage in minutes and RAM in GiBs.
If you’re running a unikernel in KVM, there’s little benefit in it over just running a Linux process. You then get to share a TCP/IP stack and so on with other processes and amortise more costs.
Ah then I misunderstood your original comment. So it sounds like you’re unhappy about the currently dominant hypervisor abstraction, that I would agree with. The interfaces for efficient resource sharing are pretty borked.
Are you talking about serverless here? Unikernels can be long-running services, at least that’s how I’ve run them in the past.
For me, the big benefit of unikernels is their ability to scale down. If you’re handling sustained throughputs of thousands of network messages per second or more, there are some benefits, but if you’re implementing services that have long idle periods of no activity and bursts of very high usage, unikernels’ abilities to either exit completely and then restart in one network round trip time or scale back to almost no resource usage is a huge win. Cloud billing systems are not set up for that at all.
I don’t really like the term serverless, because you don’t get rid of servers. The way most FaaS services are implemented today comes with ludicrous amounts of overhead. Each service starts in a lightweight Linux VM, that runs an OCI container, that runs a language runtime and some support code, which then runs the customer code. We estimated an efficient system with good hardware-software co-design to do build an attestable base with TLS flow isolation to individual services would allow at least an order of magnitude denser hosting and provide better security properties.
Thanks for the link. In your view, would the BoC as a concurrency paradigm allow for a competitive implementation for e.g. performance-critical embedded systems? Are there places where you’d say “this place needs an escape hatch to close the last few percent perf gap” or so?
I’m genuinely interested, because I was looking for things beside stackless async/await that would work well for highly constrained environments.
Possibly. BoC is really equivalent to stackless coroutines, but with a coordination layer on top and an ownership model that makes it safe. Each
whencaptures the things it need to complete and runs without the parent’s stack.The problem for BoC on resource-constrained systems is that any behaviour can spawn arbitrary numbers of other behaviours, which can exhaust memory, so you’d need to be a bit careful. It’s probably no different to avoiding deep recursion though.
You definitely wouldn’t want to use the Verona runtime for small embedded systems (it’s nice in a kernel though. I ported it to run in the FreeBSD kernel a few years ago) but I have been pondering what an embedded BoC runtime would look like.
I mean, utah2k was written nearly a quarter-century ago. (There is a certain irony of someone from Bell Labs writing this, considering the output of Bell Labs doing a lot to slow systems research…)
In the 1970s, this was addressed. Quite a few popular OSs back then treated all base I/O calls as async. Blocking was a unix thing. Out of necessity, the Berkeley unix apis on unix tried to walk that back. (They also hacked a few other unix design choices)
Programmers on those older OSs successfully built multiuser database and timesharing systems.
For simple programs, many liked the simplicity of the unix style,
IMNSHO, the underlying problem is that we decided a long time ago that the procedure call (subroutine call, function, method, etc.) is our fundamental model of abstraction. Not just for interacting with the operating system, but pretty much for everything. (“Decided” might be overstating it…it just sort of happened for various reasons, many good at the time).
I talked about this, somewhat haphazardly here: Can Programmers Escape the Gentle Tyranny of call/return?].
You can also find some of that in Mary Shaw’s Myths and mythconceptions: what does it mean to be a programming language, anyhow?, and of course many, many others have made observations about the limitations of procedure calls and proposed alternatives as the basic building blocks.
Lobsters thread: https://lobste.rs/s/vypmkr/myths_mythconceptions_what_does_it_mean
I don’t think this has been submitted here, but a piece commenting on it was submitted: https://lobste.rs/s/alzaim/thoughts_on_gentle_tyranny_call_return.
I’m not 100% convinced that concurrent-only is the way to go (and I say this as a JS dev who also uses Javascript as a good example of what concurrent-only could look like). But I agree that most languages need to choose one or the other. You either go all in on concurrency (like in Javascript, where synchronous APIs are rare and usually only exist a special-case exception alongside an equivalent async API), or you accept that concurrency is always going to be an opt-in feature with less support that will require its own way of doing things.
I’ve seen this a lot in Python, and I think we’re seeing it now again in Rust, where async support gets added, and creates a split ecosystem. You can’t mix and match, so you need to decide: am I using the sync APIs or the async ones? I wonder if there are other concurrency ideas that could mitigate this somewhat, but I think it’s just a fundamental divide. Reading files, interacting with sockets, executing processes etc are fundamental OS-level tasks, and deciding how you’re going to do those cuts to the core of a language’s standard library. In Rust, I know that some of the big proponents of the Async Way are aware of this problem, but I think their goal is to push async as the predominant way, and I’m not sure Rust is set up for that yet.
(As an aside: I know C# also was synchronous and then added the async keyword - are there similar issues there? Is there a divide between the sync and async worlds?)
The thing is, you can’t choose only one! Here’s a mundane, boring example:
You spawn a process. You want to collect its stdout and stderr. You can’t do just
because this might deadlock. The child process might interleave printing to stderr&stdout, so you have to interleave reading them. What you want to do here is to issue both read syscalls concurrently and wait for one of them to finish. So you end up writing platform specific code like this:
https://github.com/rust-lang/cargo/blob/master/crates/cargo-util/src/read2.rs
Another case you always hit when programming mundane things is that something does a
readcall, and you really want to “cancel” that read (even if it currently blocks). This again necessitates pretty horrific work-arounds with installing a process-global signal handler: https://github.com/rust-lang/jobserver-rs/commit/4c8a5985e32fa193350f988f6a09804e565f0448In the opposite direction, DNS (and, until io_uring, even file IO) — there’s just no non-blocking APIs for these!
So, I don’t think it’s up to the languages to fix this, really. It’s the OS which first needs to provide reasonable API. Only then can language designers of production languages figure out how to express the concurrency in the source code (to echo David’s comment, I suppose we might have figured how to deal with concurrency in academia, but this certainly haven’t quite percolated to day-to-day languages. Go & Rust seems like they are improvements, but also both feel quite far from optimal).
There are plenty of nonblocking DNS APIs. The gap is in the higher-level name service APIs that bundle up lots of non-DNS name databases, such as the hosts file, mDNS, NetBIOS, NIS, etc.
You can’t perfectly choose one, but I think you can abstract surprisingly well by choosing one route or the other. As you point out yourself, we’ve done pretty well pretending that things like file IO (and even memory access) are synchronous, idealised things. Yes, there are weird edge cases where the abstraction breaks, but most of the time it holds surprisingly well.
Going the fully async route brings its own problems, but it’s still in many ways just a different abstraction. Many async libraries handling file IO just run synchronous APIs on a different thread, because it’s simple, it works, and we still get the concurrency we were expecting.
So I think a lot of this is about the abstractions we chose to use on top of these sorts of APIs, and I think language designers have a lot of leeway there. Yes, the OS APIs will affect those abstractions - what works well, what works efficiently, what leaks constantly into the abstraction layer etc - but there is no shortage of interesting approaches to concurrency on top of largely synchronous APIs.
Until recently, Linux didn’t have usable async file IO, so you had to do that.
Other things David likes:
Now I’ll add to the list “Dave likes VI”
This is literally what the troll tag is for.
I didn’t consider the implication that VI is as bad as those other things, it’s not. Whatever editor makes you happy, use it.
If it helps, I use vim mode. I just think others might not know that “Dave likes X” is associated with some ideology they might not agree with. I should have spelled out that intention better.
Starting with the first post, it reads like a freedom-of-speech opinion more than “liking” Andrew Tate.
That’s a trolling, to get someone to point that out. The other links aren’t that different.
But what’s the ultimate purpose?
I really like how this site leans into the actual tech, and you typed up the kind of list that adds nothing to the tech merits of TFA, like discussing LazyVim would.
I think if you want to start cancelling someone you disagree with, using strawman lists or not, there’s X/Twitter for that.
Your statement about adding Vi to the list isn’t a troll or a joke, it’s an excuse, or so it reads.
The Bozo Bit is a hallowed tradition in hacker culture. I think it’s relevant to remind people that they might want to set the Bozo Bit on the author of an article.
The article is about the opinions of someone in tech. Sharing their other opinions is very relevant IMHO. If OP only wants to talk about VI without the association of an author they have plenty of options to do that. Sort of like how if Dave ONLY wanted to talk about freedom of speech, he could have chosen from 7.951 billion people that aren’t Andrew Tate.
I don’t think that stating things people have said publicly while citing them is “cancelling” them. I feel people use the term “canceling” when someone states an observation they don’t like. Not to mention the vast majority of “canceled” people are in fact still quite influential and very much around. I appreciate you think my words are so powerful, however I an assure you…if I had the power to cancel anyone, I think I would be the first to know.
Here’s a published, well cited research paper for you “Do [tech] Artifacts have Politics” https://faculty.cc.gatech.edu/~beki/cs4001/Winner.pdf. Spoiler alert, they do. There is no neutral, it’s all connected.
I like how this site has people discussing things that they find relevant and topical. As you noted, you can easily hide my discussion and move on if you get nothing out of it. But you commented, now again. I like that you got to share your perspective of the situation and I got to share mine. That’s what I want to see in a healthy community. To hear and be heard. That’s all I’m looking for, you don’t need to agree with me, I just want an ACK that I’m heard.
How so? That just seems like an attempt to easily dismiss people / ad hominem them. What would his opinions on Tate/Peterson/Shopify hosting Breitbart/DEI have to do with his opinions on Vim?
I appreciate you’re (probably) being facetious and making an editor wars joke when you imply we can safely dismiss his praise of Vim based on his political opinions, but all that does is open up much more contentious political wars for the sake of a joke on mostly-exaggerated editor wars; moreover, a joke that would simply fall flat to anyone who doesn’t agree with the political takes you find so obvious or finds the inverse obvious.
I don’t know the rules around discussion of politics on this site (or baiting people into such discussions), and while I’m very open to having political discussion be allowed, based on my previous experience, I don’t think that’s the kind of site that @pushcx is going for.
Edit:
Defending principles, like freedom of speech, usually entails defending the most unsympathetic people (to any given audience), since sympathetic people usually don’t become victims of violations of that principle. Nobody’s restricting the speech (or even trying) of someone whose words everyone agrees with.
While not exactly censorship, I do think using ad hominems, poisoning the well, and other such responses to get people to dismiss someone’s opinions before hearing them out aren’t exactly conducive to truth-seeking or even to free expression (especially on completely unrelated topics; if he had a track record of dumb technical takes, for example, it’d be more excusable).
Things can be both popular and silenced.
Agreed, but really, unless you can draw a line from DHH “liking” Tate — which the link you posted to prove that claim doesn’t — to Vim, then it’s off-topic, liable to bait people into further off-topic discussion which are themselves liable to be acrimonious discussions, acrimony that could have been avoided since it’s all off-topic anyway.
Flagging 12 day old comments as “off topic” and misrepresenting their contents isn’t exactly a community high road to be starting from. How did you even find this thread?
Wow, I think I touched a nerve, that’s a lot of deflections. “You’re hardly innocent, you misrepresented me and you’re flagging old content”, oh, and I detect a possible insinuation of brigading.
We can all accuse each other of misrepresenting people; you yourself were (truthfully IMO) accused due to your initial comment of misrepresenting DHH’s posts as him liking certain people, when they were clearly not him expressing a liking, or even support, of those people.
There’s no rule of the site or norm of civility to not misrepresent someone (except intentionally), because who would adjudicate that? I don’t think I did misrepresent you, but even if I did, that’s obviously unintentional, a simple matter of disagreement which has nothing to do with “community high road”.
As for the age of the submission, I don’t think 12 days is that long. I found the thread because I like to check out submissions past the front page, since I don’t check lobste.rs so frequently.
How does my alleged misrepresentation absolve you of the clearly uncivil — and possibly rule-breaking — behaviour of dredging up someone’s irrelevant-to-the-submission political opinions? Don’t try to point the finger back at me because I might have accidentally misrepresented you to deflect from a clearly uncivil post (especially one where you yourself could be credibly accused of misrepresenting others).
Librarians are some of the nicest people in the world. Librarians don’t like David.
https://blogs.library.duke.edu/blog/2023/11/30/why-were-dropping-basecamp/
I’m not a fan of Jordan Peterson, but all I can gather from that blog post is that David read his book and likes the concept of meaningful responsibility providing purpose in life. He doesn’t appear to be defending him like he does in the Andrew Tate article, just citing him for the term “meaningful burden”. I would personally leave that out of a list like this.
To what end?
I believe they were being facetious. I think it’s clear Dave is on the wrong side of history in a lot of respects.
Agreed! I just didn’t feel like it added anything at all here, so I was wondering if I was missing something (like an actual belief that this means vi too is destined to doom, or what).
I don’t think the joke is relating vi to his polemics. I’m guessing it’s a humorous way to bring up to people who are unaware that DHH has terrible opinions (because this article about vi doesn’t really make any of them clear).
I dunno, liking
viis a terrible opinion… /slol, fair enough
viis the worst form of editor, except for all the others.This statement is invalidated by the fact that emacs exists.
I think they just wanted to bring attention to the character of the author and maybe not platform them?
Sure — I suppose I expect actually saying and suggesting that would be more effective, and so the point of the flippancy is lost on me. That’s OK!
I use VIM and VIM keybindings. VI is fine. I was trying to relate my comment back to the original post in which Dave says he likes a thing. Which made me remember some of the other things he likes. I don’t like those, but I like VIM.
Thank you for the valuable on-topic links, wish your comment could be labeled
culture!These really made me reconsider Ruby on Rails as an underground railroad for political dissidents.
Are there publicly available RISCV chips which have or are planning to integrate CHERI? Since this is a MS research project, does Azure plan to roll this out in the long term?
Last I heard one will be on the market later this year. I’m sure that once David Chisnall sees this thread he’ll post additional information if there is any.
Our schedules have slipped a bit, it will be next year now, but as @jamesw says there are FPGA simulations for prototyping now.
That reminds me, I got a DE10-Lite this week, and I should look into whether I could get the FPGA version running on it.
No one has run CHERIoT Ibex on Altera FPGAs yet, but it works nicely on fairly cheap Xilinx ones. The CHERIoT SAFE project is probably a good starting point.
Here’s a press release for reference:
https://www.scisemi.com/press-release-cheriot-ibex/
There are public/FOSS systems you can run on an FPGA (or simulation) if you want to try CHERIoT today.
Anyone else wish we’d stop promoting C projects?
Why would we do that? You are always free to hide articles tagged with C.
I certainly don’t.
And why is that? Everyone is free to code in whatever they want.
I am a happy C coder until this day and posts like this shows that C is far from being dead.
C sucks, but until I have a working Rust compiler and the time to rewrite a few tens of thousands LoC in Rust, I’m still going to be interested in what’s going on in the C ecosystem.
There are some things that need to be written in a low-level language and so C or C++ (or assembly) remains the best choice. They implement core parts of the abstract machines of higher-level languages and so are intrinsically unsafe.
There are some things that were written in C a long time ago and so would be expensive to rewrite and, because they’ve had a lot of in-production testing, a rewrite would probably introduce a lot of bugs and need more testing.
And then there are new projects, doing purely application programming tasks, written in C. And these are the ones where I wish people would just stop. Even on resource-constrained systems, C++ is as fast as C and lets you build safe abstractions. For larger targets, there are a lot of better choices (often Rust is not one of them: don’t do application programming in a systems language).
No, not everything need to be Rust. There are plenty of fun you can have with C.
The fun you can have with C is the kind that leaves you wanting to take some strong antibiotics when you wake up because you don’t know what you caught.
The fun you can have with Rust is the kind that leaves you wanting to take painkillers because you had to listen to a fanboy ramble about memory and thread safety all night.
Everything was better in the old days. We should Make Programming Great Again. Where are the COBOL people when you need them?
I enjoy programming. It is still great in my opinion. Lot of good programming literature, languages and libraries are free and open source. What not to love.
😀
To clarify, I was being sarcastic! I definitely agree — we’ve come a long way in terms of language design.
Not nearly as much as the zero content “C iz bad” comments whenever anyone mentions C.
Eh, I think they’re cute. It’s a shame when C codebases are deployed in any serious context, but I’m always up for curiosity languages/libraries, especially for preserving older computer architectures. We gotta keep this stuff alive to remember where we came from.
That is certainly one of the takes of all time.
Hey, don’t shoot the messenger, I’m just echoing what a bunch of world governments and international corporations are saying.
Yeah yeah we’ve all seen the white house report. I can’t remember exactly how their guidance was worded, but I would assume it’s more along the lines of “whenever possible, in contexts where cybersecurity is a concern, prefer memory-safe languages such as Rust or Java over non-memory-safe languages such as C or C++” which is actually very reasonable and difficult to argue against.
I think there are contexts which qualify as “serious”, but where cybersecurity isn’t a concern. Also, IIRC there are some (quite esoteric, to be fair) soundness holes in Rust’s borrow checker (I think there’s a GitHub issue for one that’s been open for like 5 years and they haven’t decided on the best way to resolve it yet). Furthermore, rustc hasn’t been formally verified, and then of course there’s always
unsafecode where memory safety bugs are still possible. So I think through use of sanitizer instrumentation, testing, fuzzing, etc. you can get a C project to the point where the likelihood of it having a memory safety bug is on par with what Rust can give you. Take SQLite for example—personally I would have as much faith in SQLite’s safety as I would in any other database implementation, regardless of language.I didn’t really intend to make this some kind of language debate, but if you insist:
So you mean… just use Rust. Because Rust is essentially C with all that stuff baked in. That’s kind of the point of it. It’s C with dedicated syntax for verification tools (the ones it’s built with) to verify it, along with some OCaml control flow structures to make code more streamlined. You really can just kind of think of Rust as being C++ 2 (Not C++++ because that’s C# (that’s the pun of the name, if you never knew)).
And I’m not sure why people are still doing this “well the compiler might have a bug or two and you can manually mark things as unsafe so that means its as insecure as C” reach, it really doesn’t make any sense and nobody is accepting that as a valid argument.
The fact that Matz won’t even consider removing the GIL from Ruby saddens me so.
You keep saying that Mike, but I don’t think you quite grasp the consequence, it’s much bigger for Ruby than for Python, because you need to add a lock to all mutable objects, and in Ruby strings are mutable…
Also I’m yet to see proper benchmarks of the single threaded performance impact. They think they can reclaim the lost performance elsewhere, but that’s because Python has refused any complex optimizations for a long time, that’s not the case of Ruby.
Finally, Python’s multi-processing story is quite bad because ref counting is nasty for CoW, that’s not the case of Ruby.
I know, they said multithreading in Ruby was impossible because the ecosystem wasn’t thread safe. Sidekiq was the tool which forced the community (including Rails) to take multithreading seriously.
How do you eat an elephant? One bite at a time. My favorite timeline has Ruby aggressively adopt frozen strings and deprecate C extensions.
If you are OK deprecating C extensions, then can’t you use JRuby or another Ruby implementation? Or fork JRuby to add frozen strings?
I thought Matz even wrote another Ruby interpreter …
No. It predates Sidekiq by several years, e.g. https://yehudakatz.com/2010/08/14/threads-in-ruby-enough-already/
That totally misses the mark.
Adopting frozen strings won’t change anything, as long as a string can be mutable, Ruby has to reserve space for a lock in every String and has to check for it. For a language that’s heavily used to render HTML etc, the performance impact would be huge.
As for deprecating C extensions, it just isn’t realistic, and even if it was that wouldn’t be the only issue.
I know you don’t like it, but multi-processing for web servers and job workers works perfectly fine.
One nitpick: a blog post is not a forcing function, no one fixed thread safety bugs because of Yehuda’s post.
I had to create the
connection_poolgem in 2011 because Ruby did not have an API for sharing connections amongst threads. AFAIK Sidekiq and its infrastructure enabled the first real widespread usage of multithreading in Rails. YMMV.I linked to this post because it account of the situation at the time and ends with a good reminder:
Rails 2.3 was 2009.
I’m not sure what’s this is supposed to mean.
No offense but no. Puma predates Sidekiq, and even before that Webrick had multi-threading as an option since ages, as well as passenger (AKA
mod_railsormod_ruby) and Rainbows.Is this a backwards compatibility concern or something else?
Backward compatibility concern of course but also:
Which by the way has always been the reason for shrugging off GIL in the Python community when it was predominantly a Web language: it just wasn’t problematic. But then science and AI happened :-)
You speak of it confidently so I assume someone actually looked into it. Any chance you could share some links on the topic? Experiment results, discussions, etc.
I understand that setting/reading/waiting on a lock is an overhead but is it 3%, 30%, 300%? How much space does a lock need? Do object with lock require 2x space or is it a single byte or even bit? Can we only lock objects that are accessed cross-thread and mitigate a lot of slowdown?
I can believe that removing GIL is a big and complex task. I can believe it might not be worth the effort. But, TBH, I’m not convinced it’s not something worth looking into. And granted, I’m not the one that will probably do this work so I can’t demand anything but I’m curious, I’d like to know more about this.
It’s an hard to answer question because it depends how much a given application would need to lock. But my guess is “double digits”%.
To give you an idea, recently we had an issue in Ruby 3.4-dev where for each allocation we were hitting the ractor path, so had to do things very similar to what we need to do in the absence of GVL, including an atomic increment, and that had a very visible impact on the benchmarks we track at https://speed.yjit.org/. And that was just for allocations, dropping the GVL would mean slowdowns on most method calls.
In the case of the Python implementation it’s over 20 bytes per object (
ob_tid+ob_mutex+ob_ref_local+ob_ref_shared). That’s huge given currently the Ruby object header is 16B.It’s not so much about the effort, it’s a lot of work but doable. It’s about whether the result is desirable.
Why do you want the GIL to be removed from Ruby?
The Ruby async/multi-threading story has marked it as end of life for a long time now. Programming languages have very strong path dependence and it can be hard (or impossible) to put them on a different path.
I remember DHH being on the Django podcast and them asking about Rails’ async plans and DHH dodging that question so hard that I was reeling.
This is ridiculous… Ruby chose not to go the same route as Python with its async and sync function and instead has a unified system “a la” gevents but without monkey patching.
And Rails has supported async queries (the only part that really matters) for several years now.
Yeah this seems to be a fundamental property of software … early decisions live forever. Python is going through crazy contortions now to fix earlier architectural decisions. (We can call them “mistakes” in hindsight, but that assumes that you are able to predict the future, and should)
Reminds me of this quote from Ousterhout:
https://lobste.rs/s/gnkgum/maybe_people_do_care_about_performance#c_rognji
Not that I think Ruby should die … I think many great things have been built with it. Someone said “a benefit of choosing Ruby is that it attracts people who get things done”. And they do have a pretty good track record of successful businesses
So yeah it is too much to say “end of life”, but I’d say just “not for certain problems”, which is true of every language
Honestly I think a lot of software should die to prevent more misery from being brought into the world. Let’s please end of life PHP, salt the earth and hope people disperse and go for better alternatives.
I’ll not continue to harp on about Ruby but it’s not obvious to me that it should make the cut or will ever become a thing.
This was a great blog post, I have no idea why they deleted it.
Is this anything besides Yet Another Gnome Skin?
Completely written from scratch including the UI libraries. Rust instead of C.
I feel like we’re missing out on what we could do with an actually dynamic desktop environment (think scwm or NeWS or Emacs or Smalltalk). GNOME’s experience trying to wedge stuff in via JS points to us needing this direction, but I don’t think anyone’s taken it seriously on Linux. Probably because it requires a runtime that can compile code, introspect itself, and handle isolating and managing components, and historically only some of the Lisps provided that.
Agree. See etoileos and arcan.
I don’t have a hat for being one of the founders of Étoilé, but the project died a while ago. From my side, it was because the things that I wanted to do were really hard on existing hardware, so I went off to build the hardware that can do it. There were two important things I was trying to enable with Étoilé:
First, end-user programming at every level of the system. This meant building high-level languages that closely interoperated with low-level ones. I wanted people to be able to reuse existing things (no one should have to reimplement libavcodec / libavformat to be able to play video, for example) but in such a way that you can’t crash your program by using them wrong (or by having bugs in them). That meant that being able to sandbox them, without losing their performance, and being able to share data at the granularity of objects with them. If I started now, I might use WAsm to vaguely approximate what I want. With CHERI, I can actually build it properly.
Second, I wanted to get rid of applications as a user abstraction. I wanted a canvas, where you had different tools that could provide different objects and ways of interacting with objects. No word processor, drawing program, audio editing program, and so on. If you’ve used ClarisWorks, you’ve seen a glimpse of what I wanted: In ClarisWorks, every text field that you drew in the drawing program had the full power of the word processor. Every table you drew in the word processor had the full power of the spreadsheet. Being able to do that well required being able to compose bits of code from different sources (including random Internet people) and aggressively constrain what they can do, to the granularity of which objects that can touch, with good performance. Again, CHERI enables that.
With these, some other folks were working on things like the CoreObject data model, where you have a generic data model for tree-structured data, with extensible diffing and merging operations. This gave you unlimited branching persistent undo for free. It also gave you something that you could build live collaborative editing on top of very easily (which we did for some demos, wrapping the updates in XMPP messages and some people did in some iOS apps that used the framework), because you can extract a structured object diff and then send it to someone else who can then merge it, and you can do that every time there’s a change to the document.
The generic object model also allowed us to do some fun things for end-user programming because the view hierarchy was exposed as the same model, so you could use generic inspectors to inspect your current view hierarchy, and then inspect that, recursively. And do diffing and merging on views and build different views of the same data by using the same sort of inspectors that let you inspect the data.
To me, this more closely aligned with the Free Software ideology than an app-based model, because being able to modify, extend, and distribute programs was a core part of how you’d use the system. There wouldn’t be big monolithic and incomprehensible applications, there would be tiny tools (‘services’) that you could assemble to build rich environments. The apps-as-distinct-things model is closely tied to the economic model around selling off-the-shelf software and replicating that in a Free Software ecosystem makes no sense to me. Arcan has a similar philosophy.
So glad to get this reply to my mention of Étoilé. IMO, it’s one of those things that everybody should know about even if they can’t use it - like Smalltalk and Lisp (at least everyone here knows about those) and Forth and HyperCard.
Now, just personally, the part I cared about the most was the first part (your second paragraph) and I hope we get at least that part one day. Oh, plus I really liked the syntax and programming style you were using.
If you haven’t seen it, the STEPS project from VPRI is worth a look. They were one of our major sources of inspiration. A lot of what we were trying to do was a more pragmatic version of their system.
I have looked at some of the high level STEPS documents, but I wasn’t able to find an open source copy of the software that they wrote so that I could run it and play with it. Does this exist?
They did release some things, but I’m not sure where to find it. From my perspective, the ideas were all more interesting than the code. The code was driven by the Smalltalk mindset that it’s possible to rewrite everything in your favourite language and ignore the existence of other ecosystems, which is fundamentally limiting. As a tech demo, being able to write an entire OS and some apps in 20KLoC is nice, but it also lacks a load of features people actually want.
There’s a quote from one of the Sun founders that I haven’t been able to find a good attribution for:
It’s a lesson that a lot of Smalltalk, Go, and Rust advocates could benefit from.
I don’t think so :-(
Yes! I nearly had that on my list.
I’d like to see newspeak get more traction.
https://newspeaklanguage.org/
Oh man, you’ll just love this one book by a guy named Orwell…
I mean, I’m glad that people are doing something, but none of this really changes the “Windows 95” model of interaction we’ve all been suffering from for 30 years.
Genuinely asking, what is the “Windows 95” model, and how exactly does it cause suffering, in opposition of other “models”? What is your definition of a “UI model” and which UIs fall under what models?
Does it use being written in Rust to achieve something Gnome, Cinnamon and pals don’t? Not to suggest necessity is the only reason some new software should exist, but what’s the driving factor behind a new DE? The fluff on the homepage just reads like fluff, and doesn’t say anything substantive about the project.
I think rust projects are meaningfully easier to contribute to than C, C++, or Python ones. Good build system, one auto formatter, package manager that solves transitive dependencies problem, reasonably conveniently typed language and reasonably advanced tooling go a long way towards being able to spend less than an hour between an initial git clone and a pull request.
I’d say, among popular languages, only go is comparable to (and perhaps even better than) Rust in this respect.
So, my baseline expectation is that they’ll be able to ship more and faster than KDE and Gnome, with less work.
Compare with how quickly people were able to ship fine-grained incremental compilation in rustc, because it was implemented in Rust.
I would like to second this. I would find it hard to overstate how much easier I find it to contribute to Rust projects, despite being a novice in the language. I’ve lost a lot of time in my life trying and failing to build projects so I could contribute to them.
That first build is a big hurdle that just is usually not present in Rust (or Go) projects. The usual case is
git clone ...; cargo build. I find that’s a big convenience (and performance / memory safety / etc is just a nice bonus.)Is it still the case when you have to rely on external dependencies? Not sure about this exact project, but I would assume that cross-language projects will need complex bootstrapping either way.
Yeah, external dependencies are unsolved. But in rust you generally can avoid them. Like, for example, I believe there’s a more-or-less complete port of harfbuzz to Rust? And that’s not someone just wanting to rewrite things for the sake of it, but rather than economic observation that Rust stuff is as fast and as reusable as C stuff, but is also memory safe.
Nope, but I still find it a lot easier in most cases with external dependencies.
E.g. The Helix editor requires one external dependency, gcc. That’s not a problem to me, and my experience with building Helix was still the practically painless
cargo install.Another e.g., the most complex Rust project I’ve built was Graphite, which is a Rust and Node project. I’ve already hit npm’s pain points before, so I didn’t hit any issues.
In both cases, Cargo doesn’t interact with the other package managers.
I imagine it’s memory safety. GNOME and KDE have their own fair share of serious CVEs.
No idea, I’m happy with KDE.
But as a Rust dev, it is nice to see a good GUI library native for Rust. And what I’ve heard people have been quite happy with Cosmic.
I’d add: NVMe drives have increased drive speed by 10-100x from SSDs and spinning rust.
That’s the only advance I actually care about. Most of the things in OP are surely nice but don’t affect me personally.
Seems questionable for a content organization to gate their content behind a framework as complex as React. Will the content work 20 years from now? Plain old HTML will. I wish the NYT had a lite version like CNN: https://lite.cnn.com/
This seems like a misunderstanding of where and how React is being used. I don’t know the internals of the Times CMS (although I’ve seen it demo’d) but it either stores content as some superset of plain HTML (XML, shortcodes, whatever) or as a series of JSON objects. Either way, the content has nothing specific to React about it and translates down to plain HTML and a format their print CMS can understand through the vagaries of the system. Special interactive stories are stored as one off pages on a static site. The content in those stories is frozen at publication time and then just never touched again after an initial period of updates. The Times also has lots of weird corners you aren’t thinking about like recipes, Wordles, and Wirecutter scores that are stored in their own bespoke ways. The React of this article is much more about the chrome of the page than its content. The focus of the article was how to deal with interactive widgets being injected into the page without React (which controls the page chrome) noticing and complaining. The widgets themselves could be written in any JS framework and many of them don’t necessarily follow best practices for 2024.
Yeah, if you had a React page and you wanted to come back to it and edit it after leaving it alone for a couple of years, it would be very difficult because the source would have rotted and become unbuildable in the meanwhile. But that’s not something the Times worries about because pages are either finished projects in which case they are frozen and never go back or living projects in which case they never go stale.
My feeling is that this is a rather technical solution to fix people’s bad habits. In other languages, the fix would have been a linter that prevents you from creating jobs that run a loop, unless the only thing done within that loop was to create child jobs.
I don’t think of this as covering up bad habits so much as a new tool to help with tasks that need to be accomplished serially. A long running data migration or backfill, for example.
I have had to do this many times and while it’s not a huge deal to structure jobs to avoid the dreaded long running job, it’s nice to have the pattern ensconced in the library and not require rebuilding at every new app and/or educating engineers on how to do it.
Next step, presumably, is to take a load of money from Google to make them the default search engine and then spin up a for-profit subsidiary and then spend all of the money on things that are totally unrelated to the core mission?
Nah, that’s so last decade. Instead, we’re going with a strict “no default search deals, crypto tokens, or other forms of user monetization, ever” policy :)
Browser diversity means a lot to me. I donated $100 for the Firefox 1.0 ad in 2004, and I’m happy to donate $100 to Ladybird now, 20 years later. Good luck.
Seeing this post has unironically made me more optimistic about the future today. Obviously this is a huge undertaking but it’s sooo necessary, especially with mozilla and firefox drifting ever farther away from the interests of their users and generally falling apart.
I will be very curious to see what stance Ladybird takes on DRM in the future, jwz convincingly argues that implementing DRM was mozilla’s original sin.
The analysis isn’t particularly apt: Before EME, DRM video in browsers (including Firefox) was played using multiple proprietary plug-ins that were unconstrained in what they could do. After EME, depending on platform, DRM video is played either using a smaller and constrained proprietary plug-in or by underlying OS/device facilities. jwz is trying to make readers believe that going from multiple large unrestricted proprietary plug-ins to a smaller and more constrained one was the worst thing.
(Disclosure: I was a Mozilla rep in the HTML Working Group at the time.)
I’m not sure I find that particularly convincing. Isn’t the only argument “somebody else would have done it as a plug-in”?
Anyway, I am excited about ladybird and curious to see how they handle it. Maybe you just build a web browser without DRM and tell people to use something else for Netflix? Maybe you do add some kind of plug-in thing…? I have no idea how widevine actually works, given how powerful sidechannel attacks are I am just surprised that it isn’t entirely broken but maybe nobody cares enough.
Re widevine, it has been broken many times. It’s very simple: the CDM (content decryption module) contains a private key, the server checks that the public key you’re using is in a list maintained by Google, and then it uses the key to encrypt the content for you. This means widevine becomes a game of putting the private key in as hard a place to reach as possible and rotating it as fast as possible when it gets found.
The key is often hidden in a TPM or some other piece of hardware, but most places where Widevine is usable don’t have such a thing available, so it’s just stuck in the libwidevine.so that ships in every Chrome and Firefox. You might wonder “where could you possibly hide the key in userspace?” and the answer is “whitebox cryptography” which means creating a custom encryption function with the key compiled in and running a bunch of mathematical transformations to make it hard to get the key out of it. This is somewhat ineffective though - I believe on three separate occasions I’ve seen people talk publicly about extracting the key with a few math tricks, so I’m sure it happens way more in private.
The argument is that they had a chance to influence the W3C standard to not include DRM, and they failed to even try.
Now I am reading the linked https://web.archive.org/web/20131004213146/https://www.eff.org/deeplinks/2013/10/lowering-your-standards and it’s all quite fascinating.
The article argues at the same time that it needs to be fought and that it’s ineffective at preventing piracy which seems odd.
There are also arguments that “other protect content would follow” but afaik that hasn’t actually happened. For example the recentish WEI proposal was shot down rather quickly.
Author of that article here (please click on this link so that EFF still knows people talk about DRM: https://www.eff.org/deeplinks/2013/10/lowering-your-standards )
Not really: actually a lot of things don’t work at their stated function, and are still a problem. The problem with DRM isn’t that it stops piracy, it’s that it stops a bunch of legitimate use of your computer, and requires locking down your personal device to a degree that you don’t know what’s going on within it, can’t change it, and can’t even rely on others to reverse-engineer it to find out what is happening. Whether it works to fight piracy is irrelevant to those concerns. It just makes it extra-frustrating to try and argue against when it’s clear that it’s a dumb idea that doesn’t even work at what it aims to do.
(The reason why it was felt it doesn’t decrease piracy, btw, is that you only need one copy and a distribution mechanism to provide to everyone the pirated copy that EME is supposed to prevent.)
Really, everybody I spoke to conceded that it was less an effective anti-piracy mechanism as a contractual requirement by corporate rightsholders so far up the distribution chain that it was impossible to renegotiate. E.g. Netflix had these contracts to stream, but in order to comply with the contract, they needed some sort of DRM.
Any DRM would do: it’s not its technical capabilities that would be important, it would be primarily the legal protections and liability it provided for the rightsholders.
I think the fact that DRM’s protection is primarily legal (and IP-based) is one that downstream developers often struggle with. Like, you potentially get the legal protection the lawyers want with a XORed stream, but most technologists are going to try and build a /really good/ DRM system, even when that involves re-engineering the entire general purpose PC to lock down certain content streams. So you get all of this investment and effort to make copying locally difficult, but never impossible, creating all of these problems for real customers and adding complexity to the base platform. But pirates are just going to pull stuff out of the analog hole and then distribute it for free or cheaply anyway!
WEI was shot down, but it was also proposed, which was the problem with EME: before EME there was a dependable line the community could draw at the W3C, so that if proposals go this far, even with hardcore pressure from corporations or rightsholders, the W3C could give a principled reason why they would not except them. Now there’s precedent to override that principle. It’s much better to have a defensible line – or preferably multiple defensible lines – than to keep on having to fight the same fight, having fallen back from a previously hard line.
I think in the modern era we’re going to see an interesting shift in the conversation. For the past few years, piracy has been (perceptually) dropping online because the average person doesn’t do it – the convenience of streaming overrode any desire to get an unauthorised version. Now that streaming is increasing in price, and also fragmenting, the incentives will increase to defect, and return to sharing infringing content. I think we’re going to see a harder moral sanction against doing that, both because the zeitgeist is more in favor of shame, and because the battles over things like the DMCA are much more remote. The idea that it’s illegal for me to even describe here how to get around EME may well seem less outragious to a generation that has grown up with those kind of legal restrictions, and pervasive DRM.
Also, I’ll note that the increasing call for non-machine-readable content is going to make strange bedfellows of AI skeptics and copyright maximalists. What better way to defend your material from marauding AI bots than to wrap it up with DRM? Snakeoil salesmen don’t care who their audience is, they just change their pitch.
I still think that – like encryption – most involved technologists understand the contradictions that lie at the heart of regulating or implementing DRM. But every year we live in a DRM, locked-down, world of non-general-purpose computers, the harder it is to paint this as an absurdity.
Please fix the donation system. It refused to accept recurring donation from my MasterCard, refused one-time donation and finally started to just display “Forbidden”.
EDIT: My bank has just informed me that Donorbox are fraudulent and people have repeatedly reported being charged without their consent so they have blocked the transaction. Please use a different provider. This resulted in my card being blocked and I am being issued a new one.
Sounds like an issue with your card issuer, I’d suggest finding a more resiliant one.
I see the website says you will soon have 7 full-time software engineers on the project… that’s a lot of money! Who is the money mostly from? Why are they donating it? How do you know it will continue?
This is not to say I don’t believe the project can succeed - I just want to understand who believes in it enough to put this much money behind it, and why. Most entities don’t want to put up this kind of money without some ROI.
Looks like most of the money comes from one of the two founders/board members. While their website also lists sponsors, it does not yet seem like enough to keep things going. I really like what they set out to do here, but am still curious to know the plan for when the $1M is spent, in particular, since they say they want to grow the team further.
Essentially:
To be fair, I think we are better off with Rust than without it, and I’m glad that Mozilla funded it into existence. I’m sure that Mozilla did a lot of less convincing stuff, but they probably also did a lot of nice stuff that is less well-known but still had positive impact for the web. (For example, I suppose they played an important role in setting up the Javascript specification process in a reasonable way, asm.js is also not bad, presumably they did weight in to support open codecs and open standards, etc.)
You may know better than I do, but the vibe that I got from Mozilla for a long while was that it was a nice place to work for people who are passionate about open source and the web, and people had agency to work on things that they believed actually matters. This seems like a good way to go about things, which I think they deserve credit for. Sure, there were a few debatable decisions thrown in the mix as well, mostly dumb and values-contradicting ideas that were supposed to help make the project financially sustainable without Google backing, but I would still count Mozilla as a net positive.
(In comparison I find it harder to find excuses for Ubuntu that, while overall delivering a solid product that people want to use, seems to have a knack for make repulsive decisions all the time.)
The mozilla root program for SSL/TLS is world class too and not much talked about outside of PKI circles.
As I understand Mozilla’s involvement, Rust was largely a spin-off of Servo: The latter was a project to build a memory-safe browser engine, the former was the language built to realise that goal. Mozilla dropped funding for both. Rust managed to survive but there was quite a disruptive period when a load of key Rust contributors suddenly stopped being funded by Mozilla (2020), including one member of the Rust core team.
Servo absolutely was something that was core to Mozilla’s mission of promoting an open web. I’d love to use a web browser where the entire rendering pipeline was implemented in a memory-safe language. Mozilla fired all of the Servo team in their 2020 layoffs. They found a new source of funding last year but there was a lot of disruption.
According to Wikipedia, Mozilla started funding Rust in 2009, and the mass layoffs you are talking about happened in 2020.
From the homepage:
The entire announcement makes me happy, I’m just sad about the reason it makes me happy: That this was exactly what the original Mozilla Foundation announcement looked like 21 years ago (almost to the day) and the open web really needs a solid competitor to Chrome (just as, back then, it needed a solid competitor to IE). Mozilla consistently compromised on principles and directed funding to projects that were unrelated to their core vision (but not quite enough funding for any of those projects to actually succeed).
Yeah, I remember it too (and I continue to sigh whenever I need to e.g. remove Pocket from the toolbar of a new Firefox install and whatever else). I suppose Ladybird has the advantage of foresight of such issues that Mozilla didn’t.
If they don’t have a sustainability plan today, they will adjust their ethics when one presents itself. Happens every time. Just wait until they take VC.
They could use something like Marginalia or https://mwmbl.org/ for the default though. At least they are non-profit too.
Magical grains of sand which enable subway rides. Amazing.
I still find
testifyincredibly useful and assert that Go needs an assertion package.I also totally agree that it is the height of folly to re-invent tools from other languages (e.g. RSpec and Cucumber) in Go. The aside about Gomega and Ginko is dead on.
I’ve also heard good things about quicktest, as a smaller alternative to testify.
That looks nice enough, and the .Equal uses generics, which is good.
So a basic not-an-err check would be
qt.Assert(t, qt.IsNil(err)), compared torequire.NoError(t, err).(most of the check functions are
func abc(got, want T) Checker, which is, sure, arbitrary, bot also exactly the other way around from testify, which tend to do(want, got))I use quicktest for work, and it does the job. There are some warts though:
My “favourite”* thing related to Nginx is Avoiding the Top 10 NGINX Configuration Mistakes. To me this entire post can be summarised as “our default are bad and we should feel bad, but we’ll make you feel bad instead”.
Nginx has so many footguns that I just can’t recommend it for anything other than very simple or hobby setups anymore. It was great at being an Apache httpd alternative 20 years ago, but it’s really showing its age now.
* Meaning, wow this post should not have to exist.
Yeah, number 6 will terrify you!
No but really, Mistake 6 is “if you use an
ifstatement in the wrong kind of scope, you can get memory corruption”. That’s right, nginx.conf is a memory unsafe language. And they could 100% check for this one at config-load time, it’s entirely a static (syntactic, even) property. But they don’t, because there is no safety culture there.It’s amusing that, prior to the F5 acquisition, the Nginx wiki had a document called “If is Evil”. F5 seem to have taken that down now. Thank goodness for the Internet Archive! https://web.archive.org/web/20230802145207/https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
Oh damn, they took that down? That was one of my favorite things on the internet!
Gotta love the triple scare quotes!
What alternatives to nginx would you recommend?
It depends on the use case. Caddy is great for simple setups and gets you TLS for free. You can’t do that with Nginx. (Caddy is great for more complex setups too, but I mention simple here because you can run it with almost no configuration and have working system with TLS.)
Traefik is next on my list. Great for containerised environments as it can do auto discovery. It has a lot of tricks up its sleeve, and also has automatic TLS support. More configuration needed than Caddy.
For complex requirements, such as that described by the OP, I’d suggest Envoy. It’s a complex beast, but very powerful. At work I’m starting to replace our Nginx-based proxies with Envoy, and am writing a custom control plane for it.
For large or complex deployments I’m generally in favour of having the centralised control plane approach so that all your proxies get configured consistently. With Nginx I’m tired of having masses of templated configuration files and getting them in to Kubernetes pods.
Caddy seems popular.
I suppose this is hyperbole but if my memory is not failing me, nginx was just a blip in 2006-2007 era and was all the rage up until 2014-15, so I’d say it was absolutely great 10 years ago, I can only not comment about the state after that.
That doesn’t mean it wasn’t perfect or had no footguns, but you’re making it sound like it was a liability. It was not.
Fedora has Flatpak, which is just as bad. Back to the drawing board with all of this, whatever they were trying to accomplish it’s not worth the downsides.
I have a strong preference for Flatpak over Snap just because it isn’t hardwired to only install packages from Canonical’s repository.
It’s really not as bad. All the usual snap complaints don’t apply here: Basic system applications are not flatpaks. You can uninstall flatpak completely. Flatpak doesn’t auto-upgrade. You can downgrade flatpak apps to older versions if you want, or pin them to a specific one. Fedora doesn’t carry any “rpm that actually just installs a flatpak” packages. You can have multiple custom flatpak repository sources.
Did I miss some snap issue that’s present in flatpak and not optional?
Snaps are compressed on disk and usually need to uncompress to run which can cause significant delays in statup time. Flatpaks aren’t stored compressed and thus there is no startup delay like snaps.
This is news to me! Totally a shot in the dark and a bit naive, but couldn’t something like zram help here? I guess it’s still the decompression (and that apps can easily fill that space).
The issue is that installed snaps are still compressed disk images, with compression settings determined by the publisher (hence Canonical working with Mozilla to improve the start-up time of the Firefox snap) and very limited, publisher-dependent support for deduplication via the ability to define a stack of disk images to build an overlay filesystem from, and which must be decompressed every time you load from them while Flatpaks are installed using a “git for your filesystem” framework (OSTree) which decompresses (or hands off to your filesystem’s innate compression support which you can tune) at install time and deduplicates any hash-identical files across your entire installed base of applications.
It’s a fundamental shortcoming in Snap’s design.
Snap just generally does a lot more lumping of concerns together into a single system than Flatpak, where they try to focus on well-factored components. (eg. Snap is trying to be both a Flatpak competitor and a system for immutable distros. Flatpak is just an integrating wrapper on top of the OSTree system that immutable distros like Fedora Silverblue also use for system packages via rpm-ostree. Likewise, Flatpak’s sandboxing is a separate tool you can use independently named bubblewrap.)
It’s that an issue? If you want to save space, you can mark the whole flatpak store for compression (if you’re using btrfs or zfs) and you’ll gain both quick startup and disk saving
I currently run Pop!_OS, a Debian-based distro that ships with flatpak. Their app store includes both flatpaks and debs, and doesn’t make a clear distinction between them. I always curse them when I click the wrong one.
My main gripes with Flatpak are:
I had to migrate off fedora when they dropped X11 support (at least for KDE) on this last version (40).
I won’t say I’m happy with Kubuntu but at least it just takes a minute to fully disable snap!
KDE itself is on the verge of dropping support for X11. The latest release’s release notes made it very clear that new feature development is happening in the Wayland version now.
As an outsider, it’s interesting to me that wayland/X isn’t fully hidden behind a Qt abstraction.
It is hidden, but the Plasma Shell itself has to display the desktop to you somehow and being a display server is just not something Qt can do.
Also there are Qt apps which intentionally disable Wayland support. AFAIK Krita specifically disables it because X has better graphics tablet support.
it is, but that doesn’t help me when i’m under wayland and a feature I rely on doesn’t work because Wayland plain does not support it (last one in date : moving the mouse cursor to an arbitrary position from the app to implement infinite scrolling - this works on macOS, Windows and X11 but Wayland just doesn’t allow arbitrary apps to do this at all which makes a basic GUI operation in any pro audio/video software a PITA instead)
I’m an X11 holdout but I’m wondering if I’m going to have to actively engage with fixing Wayland at some point (or if some new thing might come along and displace it before we’re done reimplementing X…)
There is Xwayland, but I don’t think it’s capable of running some window managers. I’m also not clear on what limitations it has.
I’m also an X11 holdout.
In the last couple of months, we’ve been seeing work on improving rootful XWayland so, if the necessary user-empowering features aren’t ready by the time LTS distros drop bare-metal X11, it should be possible to do something like running a kiosk-oriented Wayland compositor which runs rootful XWayland as its single application, then dig up something like
kwin_wayland’s option to run with an X11 backend, but with support for rootless operation.If you’re writing software against Qt directly, or KDE’s libraries, it usually is. Kwin itself is an implementation of a display manager, so obviously has to care quite a bit about the underlying protocol.
What I read is that they want to continue supporting X11 for the forseeable future. And I’m glad they do. Should that change, Linux desktop is in serious trouble.
I don’t need new features. But I need my computer to work. I need to be able to record my screen, share my screen, setup global hotkeys, simulate inputs… And I need to do that in a standard DE-agnostic way!
So X11 it is! And if one day KDE drops X11 support, I’ll be forced to drop KDE.
I’ll gladly let more enthusiastic people beta-test this one for me. Maybe one day Wayland will be usable (and no, last I checked was less than a month ago! I know it doesn’t work for me).
I don’t want to sadden you, but plenty of DE’s support for X11 is already minimal, even bugs get low priority in fixing. I know this is the case for Gnome, for example.
No worries, I’ll be fine, I have a feeling I’m far from being the last X11 user
Is there a good write up of the criticism?
I also vastly prefer flatpak to snap. Here’s a decent comparison table from the AppImage project.
I don’t have one handy, my main criticism is that I don’t want to wait 20 seconds for an app to “boot up” I don’t care what the benefits are.
Interesting, my experience with Flatpak on Fedora is that applications open in about 2 seconds. This is specifically on Aurora, where just about every application is Flatpak’d. Applications taking 20 seconds longer to start up under Flatpak than natively sounds like a Flatpak bug, not expected behavior.
I run Pop!_OS, which is a debian-based distro that uses flatpak.