I have no idea what counts as a credible source in C++-land. The C++ standard and all these “authoritative sources” appear more and more like broadcasts of random string generators.
I suspect his corrected version still has a problem.
His first issue was that the next day’s directory might be created early… giving a window on the order of ~5 minutes in which it could erroneously be processed and marked complete. The fix was to mark only previous day directories as completed.
I would like to point out that there is also a race condition around the precise moment when the date changes. We know it takes on the order of ~1 minute to read a directory, so depending on how the code is written there might be a ~1 minute window or perhaps only a ~few milliseconds window in which this application and the other applications that are writing records might disagree about what the date is. Either way, it leaves a possibility for imperfect data.
Given that skipping historical directories is an optimization (and the system worked without that optimization in place), I would probably switch to marking directories as “completely scanned” only if they are older than yesterday (instead of older than today). That means we will continue re-scanning yesterday’s directory for a full day, but it also gives us a full 24 hours to be confident that all systems agree the date has changed. (If the optimization were more important, I might give it 30 mins instead of 24 hours – but certainly something much larger than the expected run time of any processes that probably cache the date.)
For those who didn’t read the article, here is the final line: “Maybe we need a new FAANG acronym that covers OpenAI, Anthropic, Google, Meta and Amazon. I like GAMOA.”
Because they haven’t produced their own GPT-4 class model yet. I like the Phi series but they’re not at the same level as GPT-4o/Claude 3.5 Sonnet/Gemini 1.5 Pro yet.
The insights from someone who has serious experience is incredibly valuable.
But I find myself a bit perplexed. I fully understand the complaint (of Haskell) that there are many ways to solve each problem and these sometimes don’t work together or make your build chain and tooling unstable. I fully understand the complaint (of Haskell) that error messages are unhelpful.
I do not fully understand what appears to be the author’s largest complaint: that there are several libraries available to solve most problems and so a programmer is forced to choose. Why allow this to be a drag? Let someone publish their list of library preferences. Or pick the library that DOESN’T require you to recompile your IDE. Or pick the one that is first in alphabetical order. I don’t see why more choices is harmful. (Except perhaps if fragmentation of effort means that none of the libraries is actually complete.)
I’m sure there IS something behind this claim. What am I missing here?
In my personal experience, choosing a library for an industrial project comes with the implicit responsibility of choosing the “best” one—or, at least, not a clearly inferior one—per some calculus of “best.” Further, the decision process might go into an ADR or similar documentation, just depending. These qualifying factors include things like assessing the amount of the library’s API churn, or the primary library’s author history of abandoning projects. How long might this library be maintained (think Lindy effect)? Is there a big semver major bump coming up for the library such that its documentation is dithered and you’ll also need to go water it in a month? Is its interface layer ergonomic for your anticipated use cases? Do the library’s motivations make sense, and are they aligned with your goals (ex. performance, simplicity, API alignment with a framework that you’re using/not-using, etc.)? Yes, I understand that libraries should be a black box, but if you think you’ll potentially reach or exceed the boundaries of the library’s capabilities, then you might want to assess the readability of its code (and/or perhaps check its test suite). Is its documentation robust, yet also commensurate, length- and tone-wise, with the library’s impact or importance for your project? Has the primary corporate sponsor changed a project’s license for a similar library in the past? What are the general vibes of the main authors/maintainers (ex. ghost, dilettante, star-grabber, well-known serious industry veteran, algorithm wonk, etc.)? Then there’s baseline stuff like licensing and “is it performant?”.
So I would say if I’m faced with one single JSON library that I need in an ecosystem that seems roughly right, I would circle the car real fast, kick the tires, grab the keys and drive it off the lot. If I’m in a different ecosystem and faced with 10 JSON libraries, then I know that I’m going to have spend time doing at least some of the vetting mentioned above.
Less serious projects have lower stakes, so calibrate effort accordingly.
The drag I found was that there’d be a handful of incompatible libraries defining some core data structures within a field, and the rest of the libraries would be pretty evenly split on which one they depend on. Then you end up doing manual dependency resolution, rewriting things a dozen times out of indecision, reimplementing functionality you know already exists but can’t use, trying some horrible lossy conversion between the competing ecosystems, and so on.
IIRC I found this in audio, MIDI, 2D/3D graphics, and probably more.
Oh! That seems to me to be a much stronger case than the other reasoning I have heard. It’s not arguing that I need to expend time worrying about which library to use (which could be eliminated by simply picking one and foregoing the benefits of all the others). This illustrates that multiple libraries can be strictly inferior to having a single library, because it causes splits in OTHER libraries that depend on it.
More generally, RAII is a feature that exists in tension with the approach of operating on items in batches, which is an essential technique when writing performance-oriented software.
This is a falsehood some people in the intersection of the data oriented design and C crowd love to sell. RAII works fine with batches, it’s just the RAII object is the batch instead of the elements inside. Even if the individual elements have destructors, if you have an alternative implementation for batches C++ has all the tools to avoid the automatic destructor calls, just placement new into a char buffer and then you can run whatever batch logic instead. Don’t try bringing this up with any of them though or you’ll get an instant block.
I do highly performance sensitive latency work with tighter deadlines than gamedev and still use destructors for all the long lived object cleanup. For objects that are churned aggressively avoiding destructor calls isn’t any different than avoiding any other method call.
Agreed, this post is making some wild claims that don’t hold up in my experience. I’m writing a high-performance compiler in Rust, and most state exists as plain-old datatypes in re-usable memory arenas that are freed at the end of execution. RAII is not involved in the hot phase of the compiler. Neither are any smart pointers or linked lists.
I simply find the argument unconvincing. Visual Studio has performance problems related to destructors => RAII causes slow software?
“Exists in tension” seems accurate to me. Yes, you can do batches with RAII, but in practice RAII languages lead to ecosystems and conventions that make it difficult. The majority of Rust crates use standard library containers and provide no fine grained control over their allocation. You could imagine a Rust where allocators were always passed around, but RAII would still constrain things because batching to change deallocation patterns would require changing types. I think the flexibility (and pitfalls) of Zig’s comptime duck typing vs. Rust traits is sort of analogous to the situation with no RAII vs. RAII.
I think it’s the case that library interfaces tend not to hand control of allocations to the caller but I think that’s because there’s almost never pressure to do so. When I’ve wanted this I’ve just forked or submitted patches to allow me to do so and it’s been pretty trivial.
Similarly, most libraries that use a HashMap do not expose a way to pick the hash algorithm. This is a bummer because I expect the use of siphash to cause way more performance problems than deallocations. And so I just submit PRs.
Yes. I write Zig every day, and yet it feels like a big miss, and, idk, populist? “But don’t just take my word for it.” Feels like too much trying to do ‘convincing’ as opposed to elucidating something neat. (But I guess this is kind of the entire sphere it’s written in; what does the “Rust/Linux Drama” need? Clearly, another contender!)
It doesn’t, but without it I don’t really see the post offering anything other than contention for the sake of marketing.
I spend somewhere between 2 to 8 hours a day working on my own projects. (“2” on days I also do paid work, but that’s only two days a week.) Zig has been my language of choice for four or five years now; you can see a list on my GitHub profile. A lot of my recent work with it is private.
Thank you! I really like it, and I’m a little sad that Rust — which I still use often, maintain FOSS software in, and advocate for happily! — has narrowed the conversation around lower-level general-purpose programming languages in a direction where many now reject out of hand anything without language-enforced memory safety. It’s a really nice thing to have, and Rust is often a great choice, but I don’t love how dogmatic the discourse can be at the expense of new ideas and ways of thinking.
I very much agree. A Zig program written in a data-oriented programming style, where most objects are referenced using indices into large arrays (potentially associated to a generation number) should be mostly memory safe. But I haven’t written enough Zig to confirm this intuition.
I don’t remember the arguments against RAII much (has been a few years since) but that Zig doesn’t have RAII feels like an odd omission given the rest of the language design. It’s somewhat puzzling to me.
Hm, it’s pretty clear RAII goes against the design of Zig. It could be argued that it’d be a good tradeoff still, but it definitely goes against the grain.
Zig requires keyword for control flow. RAII would be a single instance where control jumps to a user defined function without this being spelled out explicitly.
Zig doesn’t have operator overloading, and, more generally, it doesn’t have any sort of customization points for type behavior. «compiler automatically calls __deinit__ function if available” would the the sole place where that sort of thing would be happening
Idiomatic Zig doesn’t use a global allocator, nor does it store per-collection allocators. Instead, allocators are passed down to specific methods that need them as an argument. So most deinits in Zig takes at least one argument, and that doesn’t work with RAII.
I was unaware that Zig discourages holding on to the allocator. I did not spend enough time with Zig but for instance if you have an ArrayList you can defer .deinit() and it will work just fine. So I was assuming that this pattern:
var list = ArrayList(i32).init(heap_allocator);
defer list.deinit();
Could be turned into something more implicit like
var list = @scoped(ArrayList(i32).init(heap_allocator));
I understand that “hidden control flow” is something that zig advertises itself against, but at the end of the day defer is already something that makes this slightly harder to understand. I do understand that this is something that the language opted against but it still feels odd to me that no real attempt was made (seemingly?) to avoid defer.
But it very much sounds like that this pattern is on the way out anyways.
Zig’s std.HashMap family stores a per-collection allocator inside the struct that is passed in exactly once through the init method. Idk how that can be considered non-idiomatic if it’s part of the standard library.
Zig is a pre 1.0 language. Code in stdlib is not necessary idiomatic both because there’s still idiom churn, and because it was not uniformly audited for code quality.
As someone who doesn’t use Zig or follow it closely, both the fact that that change is being made and the reason behind it are really interesting. Thanks for sharing it here
Even if the individual elements have destructors, if you have an alternative implementation for batches C++ has all the tools to avoid the automatic destructor calls, just placement new into a char buffer and then you can run whatever batch logic instead.
I’ve never used placement new, so I don’t know about that, so my question is, how do you do that? Take for instance a simple case where I need a destructor:
If I have a bunch of elements that are both constructed at the same time, then later destroyed at the same time, I can imagine having a dedicated Element_list class for this, but never having used placement new, I don’t know right now how I would batch the allocations and deallocations.
And what if my elements are constructed at different times, but then later destroyed at the same time? How could we make that work?
Don’t try bringing this up with any of them though or you’ll get an instant block.
I think I have an idea about their perspective. I’ve never done Rust, but I do have about 15 years of C++ experience. Not once in my career have I seen a placement new. Not in my own code, not in my colleagues’ code, not in any code I have ever looked at. I know it’s a thing when someone mentions it, but that’s about it. As far as I am concerned it’s just one of the many obscure corners of C++. Now imagine you go to someone like me, and tell them to “just placement new” like it’s a beginner technique everyone ought to have learned in their first year of C++.
I don’t expect this to go down very well, especially if you start calling out skill issues explicitly.
I’ve never done Rust, but I do have about 15 years of C++ experience. Not once in my career have I seen a placement new. Not in my own code, not in my colleagues’ code, not in any code I have ever looked at.
I’m a little bit surprised, because I’ve had the opposite experience. Systems programming in C++ uses placement new all of the time, because it’s the way that you integrate with custom allocators.
In C++, there are four steps to creating and destroying an object:
Allocate some memory for it.
Construct the object.
Destruct the object.
Deallocate the memory.
When you use the default new or delete operators, you’re doing two of these: first calling the global new, which returns a pointer to some memory (or throws an exception if allocation fails) and then calling the constructor, then calling the destructor. Both new and delete are simply operators that can be overloaded, so you can provide your own, either globally, globally for some overload, or per class.
Placement new has weird syntax, but is conceptually simple. When you do new SomeClass(...), you’re actually writing new ({arguments to new}) SomeClass({arguments to SomeClass's constructor}). You can overload new based on the types of the arguments passed to it. Placement new is a special variant that takes a void* and doesn’t do anything (it’s the identity function). When you do new (somePointer) SomeClass(Args...), where somePointer is an existing allocation, the placement new simply returns somePointer. It’s up to you to ensure that you have space here.
If you want to allocate memory with malloc in C++ and construct an object in it, you’d write something like this (not exactly like this, because this will leak memory if the constructor throws):
template<typename T, typename... Args>
T *create(Args... args)
{
void *memory = malloc(sizeof(T));
return new (memory) T(std::forward<Args>(args)...);
}
This separates the allocation and construction: you’re calling malloc to allocate the object and then calling placement new to call the constructor and change the type of the underlying memory to T.
Similarly, you can separate the destruction and deallocation like this (same exception-safety warning applies):
In your example, std::unique_ptr has a destructor that calls delete. This may be the global delete, or it may be some delete provided by Foo, Bar, or Baz.
If you’re doing placement new, you can still use std::unique_ptr, but you must pass a custom deleter. This can call the destructor but not reclaim the memory. For example, you could allocate space for all three of the objects in your ‘object’ with a single allocation and use a custom deleter that didn’t free the memory in std::unique_ptr.
Most of the standard collection types take an allocator as a template argument, which makes it possible to abstract over these things, in theory (in practice, the allocator APIs are not well designed).
LLVM does arena allocation by providing making some classes constructors private and exposing them with factory methods on the object that owns the memory. This does bump allocation and then does placement new. You just ‘leak’ the objects created this way, they’re collected when the parent object is destroyed.
I’ve done very little systems programming in C++. Almost all the C++ code I have worked with was application code, and even the “system” portion hardly did any system call. Also, most C++ programs I’ve worked with would have been better of using a garbage collected language, but that wasn’t my choice.
This may explain the differences in our experiences.
Yup, that’s a very different experience. Most C++ application code I’ve seen would be better in Java, Objective-C, C#, or one of a dozen other languages. It’s a good systems language, it’s a mediocre application language.
For use in a kernel, or writing a memory allocator, GC, or language runtime, C++ is pretty nice. It’s far better than C and I think the tradeoff relative to Rust is complicated. For writing applications, it’s just about usable but very rarely the best choice. Most of the time I use C++ in userspace, I use it because Sol3 lets me easily expose things to Lua.
I think it very much also depends on the subset of C++ you’re working with, at a former job I worked on a server application that might have worked in Java with some pains (interfacing with C libs quite a bit), and in (2020?) or later it should have probably be done in Rust but it was just slightly older that Rust had gained… traction or 1.0 release. It was (or still is, probably) written in the most high-level Java-like C++ I’ve ever seen due to extensive use of Qt and smart pointers. I’m not saying we never had segfaults or memory problems but not nearly as many as I would have expected.
But yeah, I think I’ve never even heard about this placement new thing (reading up now), but I’m also not calling myself a C++ programmer.
Placement new is half the story, you also need to be aware that you can invoke destructors explicitly.
A trivial example looks like
char foo_storage[sizeof(foo)];
foo *obj = new (&foo_storage[0]) foo();
obj->do_stuff();
obj->~foo(); //explicitly invoke the destructor
If you want to defer the construction of multiple foos but have a single allocation you can imagine char foos_storage[sizeof(foo)*10] and looping to call the destructors. Of course you can heap allocate the storage too.
However, you mostly don’t do this because if you looking for something that keeps a list of elements and uses placement new to batch allocation/deallocation that’s just std::vector<element>.
Likewise if I wanted to batch the allocation of Foo Bar and Baz in Element I probably would just make them normal members.
class Element
{
Foo foo;
Bar bar;
Baz baz;
};
Each element and its members is now a single allocation and you can stick a bunch of them in a vector for more batching.
If you want to defer the initialization of the members but not the allocation you can use std::optional to not need to deal with the nitty gritty of placement new and explicitly calling the destructor.
IME placement new comes up implementing containers and basically not much otherwise.
Note that since C++20+ you should rather use std::construct_at and std::destroy_at since these don’t require spelling the type and can be used inside constexpr contexts.
You likely use placement new every day indirectly without realizing it, it’s used by std::vector and other container implementations.
When you write new T(arg) two things happen, the memory is allocated and the constructor runs. All placement new does is let you skip the memory allocation and instead run the constructor on memory you provide. The syntax is a little weird new(pointer) T(arg). But that’s it! That will create a T at the address stored in pointer, and it will return a T* pointing to the same address (but it will be a T* whereas pointer was probably void* or char*). Without this technique, you can’t implement std::vector, because you need to be able to allocate room for an array of T without constructing the T right away since there’s a difference between size and capacity. Later to destroy the item you do the reverse, you call the destructor manually foo->~T(), then deallocate the memory. When you clear a vector it runs the destructors one by one but then gives the memory back all at once with a single free/delete. If you had a type that you wanted to be able to do a sort of batch destruction on (maybe the destructor does some work that you can SIMD’ify), you’d need to make your own function and call it with the array instead of the individual destructors, then free the memory as normal.
I’m not trying to call anybody out for having a skill issue, but I am calling out people who are saying it’s necessary to abandon the language to deal with one pattern without actually knowing what facilities the language provides.
There are different ways you could do it but one way would be to have a template that you specialize for arrays of T, where the default implementation does one by one destruction and the specialization does the batch version. You could also override regular operator delete to not have an implementation to force people to remember to use a special function.
So, how are people supposed to prove, over the Internet, that they’re blind, and not pretending to be blind for fraudulent purposes? And/or: How can the other side accurately determine blindness across the Internet?
If neither of those things can be done at any reasonable rate of success, what is to do be done? It seems like a hard problem to solve.
how are people supposed to prove, over the Internet, that they’re blind, and not pretending to be blind for fraudulent purposes?
They should not be expected to prove that. Instead, websites should offer accessibility to the blind where possible (through things like screen readers) for any user who wants it, not just for blind users.
Maybe I just had eye surgery and I’m not supposed to view screens for the next 2 weeks. Maybe I just feel like lying in bed with my eyes closed and browsing your website. Why should you care?
I’ve been saying it for a long time that accessibility is for everyone, not only for people who can’t use the UI without it at all. Screen magnification and high-contrast themes are essential for people with serious vision impairments, but they are still great if one’s eyes are merely tired.
I have a friend who uses magnification in the browser and the terminal emulator as an alternative to wearing glasses all the time. Would anyone dare to say that one needs to prove that they are legally blind and already wear the strongest optically feasible glasses before they can also use screen magnification?
People shouldn’t have to prove that they’re disabled or be subjected to any form of test from the other side in order to be provided accessible access, for any product, ever. This is especially true before the person asking for access has done anything that could be construed as actual fraud (ie prematurely defensive design is offensive by design).
I don’t think that’s what they should be proving at all. We ask users who aren’t blind to do a thing that we believe demonstrates that they’re human as opposed to a computer program attempting to simulate a human’s attention.
We should still be trying to demonstrate the same thing for people whose physical differences make it impossible for them to complete our automated tests. It might be more labor intensive for the site operator to vet them in a less automated way, but the operator should be looking to establish that the user is a real human, not to establish the user’s blindness or lack thereof.
Sure it’s more manual and more costly for that kind of demonstration, but businesses are required to provide that sort of accommodation, and it’s part of the cost of preventing bots, if you’ve decided your business needs to prevent bots.
Exactly, and my point is that it’s not right for hCaptcha to make it blind people’s problems that they find it difficult to prove that someone is blind.
If Michael Taboada (the author) were inclined to do so, he could contact the owners of websites based in USA that use hCaptcha. He could reach out to their legal team and let them know that since he, as a blind person, was banned from using their website, that he was in a position to sue them under the ADA. He could stress that he didn’t INTEND to sue them, but that he was reaching out as a favor to let them know about their liability.
Many companies would react very strongly to this. And hCaptcha might respond differently to their customers than they do to an individual who they seem to assume is trying to hack their system.
I think any compiler that wants to self-host ought to provide something like this: a chain of builds which can be used to prove that it is free of Reflections on Trusting Trust issues.
I think that the ideal way to store this chain of builds would be:
(1) A chain of build scripts, rather like what Motiejus Jakštys did here, which can be run in series to recreate the current build. The difference being that the compiler team provides the list of build scripts instead of Motiejus having to recreate it with help from Hilton Chain.
(2) The binary from each stage. Storing this enables the chain to be verified probabilistically WITHOUT needing to build the whole thing – if any link can easily be verified with a single build, then one can randomly sample a few links to confirm with reasonable confidence that if we NEVER see a failure-to-reproduce, then the whole chain must be reliable.
Sadly, no. (I originally planned to write that we needed to store only the hashes, but then realized why it wouldn’t work.)
If we stored only the cryptographic hash at each stage then a verifier could not verify a single link in the chain… they would need the artifact from the previous stage which could only be obtained by starting from the beginning and running all the way through.
Do you think we could just use git tree hashes as our references and store all of this in the one repo? Git stores the whole Merkle chain/tree and can be pretty easily verified…
(1) I’m working to set up the polling place. I’m in the US and we vote on Tuesday. I am also responsible for running my local voting location. My innovation this year is to invite local KIDS to cast ballots for whether to hold a “campfire” or a “board game night” in ~2 weeks – then I’ll hold that event for the community.
(2) In any free time I’ll be trying to make as much progress as possible on a coding project for work. We’re establishing a new framework and I want to lay out a lot of best-practice examples before more than a few team members are working in it.
While I agree that online conferences differ from the on-site ones, I think that what’s not taken into consideration is that most of on-site conferences are not accessible to a lot of people and tend to be quite expensive. It’s fun that the author has a choice to decide which one to attend, many people don’t have that choice and still want to participate to conferences.
I believe most JVMs use a second return register that the branch on after calls to jump to catch and cleanup calls. VMKit originally used the Itanium unwind model and it was far too slow in Java where exceptions are common.
The way I’d like to implement it is to return either a value or an exception object in the return register and use the carry flag as the discriminator, so each call (to a possibly throwing function) is a call followed by a branch on carry. Most CPUs statically predict branch-on-carry as not-taken and so this is basically free (tiny increase in I-cache usage) in the no-exception case, but also fairly cheap (pipeline bubble from incorrect speculation) if an exception is thrown.
If no error occurred, and the desired action was performed, the SWI routine will clear the ARM’s V (overflow) flag on exit. If an error did occur, the SWI routine will set V on exit. Furthermore, R0 will contain a pointer to an error block, which is described below.
This is the BSD system call calling convention as well. Return values are either integer values (or, in a very small number of cases such as mmap, pointers) or errno values. If the carry flag is set, they’re errno values. After the syscall / SWI instruction, you can branch on carry to the routine that sets errno.
On MIPS and RISC-V, I think a second return register is used to get the same thing, but that doesn’t play as nicely with the branch predictor (and precludes exposing system calls that return two things or errno).
Linux instead requires that all system calls return negative errno values or non-negative success values.
I find there is some insight in the observation at the top that events “trigger actions” and they “carry data” and that these two different roles need to be respected.
In fact, I have some design issues right now within my organization where there is a conflict between some groups who mandated that events always contain exactly all the fields within the domain, and others who complain that (1) this means there isn’t a field telling “what happened” and (2) that makes everything slow because it must go to several different systems to collect all of that data.
I have been making the argument that the single contains-all-the-fields event was impractical; perhaps I should instead have been making the argument that there are two separate use cases, and for the “trigger action” use case we should have fewer fields.
Notice that this is DIFFERENT from what Lutz Hühnken (the author of the piece) concludes. He concludes that you should reach a happy medium: include the trigger information plus whatever “wider” data is easy. My own takeaway is you should have two events: a trigger event and a wide event. (And, of course, feel free to skip either one if it won’t actually be needed.)
This is awesome feedback, thanks! Learning that a) others have similar experiences and b) came to other (maybe better?) solutions alone was worth writing it down.
Not a flutter developer but shame it had to come to this. Google could have tapped into a bunch of skilled volunteer contributors. Instead PRs and bug reports are just being left to stagnate. If this gets traction hopefully it’s a wakeup call for Google.
Increasingly does seem short-sighted to trust Google with the stewardship of anything these days. I can’t guarantee they won’t randomly kill off half of Google cloud tomorrow, let alone keep a framework on life support.
We definitely have a long backlog of issues and feature requests. We’re the victim of our own success, we don’t have the resources to fix every single bug or add every feature. That’s one of the benefits of open-source though, if something is important to your organization, you can contribute!
However, we take PRs and contributions seriously. We have triage processes to review contributions within a week. If you have examples of PRs stagnating, please let us know. This would be unexpected and something we should fix.
Why is this buried several levels deep in the conversation?
Ultimately, either the Flutter team has the resources to review contributions within a week, or they do not have the necessary resources. If they DO have the ability to do timely reviews of contributions (and a willingness to accept properly-written contributions), then no fork is needed, although perhaps there IS a need for a group of qualified contributors outside of Google to review issue reports and feature requests and produce working contributions. If the Flutter team does NOT have the ability to do timely reviews of contributions then they either need to cooperate with someone who does have that time, OR a fork is needed.
Trying to capture the good practices in systems instead of relying on heroes is very sound advice.
The article goes further and says “If you have strong enough rules, you don’t need to worry about the skills of the programmers.”
I don’t think that is at all accurate, and I don’t think the article provides any support for it. An example they provide is this: “One of the solutions was to introduce a rule to memoize everything returned by hooks, to automatically prevent future problems with memoization.”
I can easily believe that most of the things returned by hooks should be memoized. I am skeptical of the claim that ALL things returned by hooks should be memoized. So a library or tool which automatically memoized the results with a way to annotate that some should be omitted might be a great idea. Hiring unskilled developers because the memoization is automatic is NOT a good idea in my experience.
One of the items is “Test all the fucking time!”. I have noticed that when I (as a veteran developer with ~30 yrs experience) sit down with a relatively new developer one of the glaring differences is that I write a few lines of code then test it; they often start by writing a page of code.
Another difference (not mentioned in this article) is that I spend a lot of time writing specifications for boundaries – things like a comment defining how a function is designed to behave, while new developers often just dive into the code first and only think about the interfaces afterward (if at all). This is a little more than Rodrigo’s “Packaging matters”.
Interesting. I definitely try to test the code I write, but I have rarely been in an environment where writing unit tests for everything is feasible. I mostly test locally whatever I can and manually test on development. In my experience this is more efficient than trying to test everything exhaustively, since testing exhaustively means I need to write mocks, use dependency injection, etc. This takes time, makes the code more complicated, and often the bugs are in the integration between services anyway: If a test using a mock service passes it only really says the code does what you expect in this particular case if the service behaves exactly the way you expect it to.
A notable exception is code that is critical and/or complex. Most of the code I work on nowadays is simple boilerplate stuff that reads/writes from multiple data sources, and writing tests for that will probably take longer than writing the code itself. Also, if you get it wrong, it’s easy to see and fix. I also worked at a compiler company, where most of the changes would fall in this category: I simply would not think of pushing a change without multiple unit tests.
In the end all “rules” for programming boil down to “use common sense”.
I have rarely been in an environment where writing unit tests for everything is feasible
When I say I “test” after a few lines of code, I don’t necessarily mean unit tests. In some environments that is the right tool, in other environments “run it under the debugger” or “use the repl” might be it. I just want to something to confirm that things are behaving as I expect them to, and I want to do that as soon as I have completed a single “thought”, not after I have created a whole structure.
Most of the code I work on nowadays is simple boilerplate stuff
That rings true for me also. The simpler the the code, the more of it I test in a single bite.
this falls pretty squarely in the category of “technically true but missing the point”.
Is Go’s panic/recover mechanism fundamentally exception-like in nature? Yes, it is. Are there codepaths that exist in which panic/recover is used for error handling? Yes, there are.
Those cases are few and far between, they’re not the dominant and main mechanism for error handling. The stdlib and the ecosystem uses the error interface and multiple return for error-handling in the vast majority of error-handling codepaths, to such a degree that it’s conventional wisdom that a package should not panic and expect that a caller call recover().
It would be strictly more accurate to say “in Go, errors are nearly always signalled through values that use the error type, which is an interface, which is a boxed fat pointer on the heap, and represents a node in a tree of error values that is traversable using the functionality provided by the errors package, except in a few situations where they use panic and recover, but generally speaking it’s unlikely you’ll ever encounter a library that calls panic and expects you to recover”, but that’s a lot of words, so we just say “Go doesn’t use exceptions for error handling”, because for nearly every person in nearly every situation that’s the useful level of accuracy that people need when they are learning the language and not fretting about every corner case up front.
Go and the ecosystem usually don’t focus on correctness, and are fine with 90% solutions. So the fact that there’s a corner case (bar() panics) wouldn’t be a concern.
You’ll just get a crash and a new K8s pod, so don’t worry about it ;)
I think ordinary Go code doesn’t run in critical contexts, so that attitude is mostly fine in practice. You just don’t want to use Go outside its target/niche.
From a design perspective, Rob Pike infamously (basically) said Googlers are not to trust with anything advanced like generics or iterators; that’s for researchers.
And simplicity is the Go ethos, even at the cost of trading off correctness in some cases.
I personally find that mentality condescending and sad as a programmer that likes to perfect their craft, and robust code, even if just to explicitly acknowledge something’s not handled.
That being said Go is not all bad, my favorite things about it are cooperative cancellation via Context, and how tooling just works*.
I do have a lot of critical opinions about it because I’ve used it a lot and my values don’t align.
Yes recover() is sprinkled at appropriate boundaries, such as request handling for net/http, and that’s definitely a good thing.
There’s also cases where you can’t recover(): some runtime behavior is a “fatal error”, not a panic. For instance unlocking an unlocked mutex, or concurrent writes to a map.
I’m not against panic at all, it makes sense in most languages, and definitely in Go’s target use cases.
That doesn’t conflict with my opinion that “usually don’t focus on correctness” is a fair assessment. The rest of my comment is maybe a bit too snarky because I was feeling jokey.
So I guess that means the problem is that the code doesn’t set x.a back to what it was originally, as though in a database transaction, in the case that bar() panics?
If bar() panics, in 99% of cases we should be letting the program crash so partial update of an object doesn’t really matter anyway. Go is being pragmatic here.
This was my thought as well. Correct me if am wrong but shouldn’t we be aware enough to handle this case when we implement it like that? Or the requirement is that the compiler should tell it?
“Go doesn’t use exceptions for error handling” is a lie, regardless of how convenient of a lie it is. A newcomer to the language is building a knowledge foundation that is wrong if presented with such statement.
Just say that there are exceptions but should almost never be used because reasons X Y Z.
I love the audacity of writing “I will not try to motivate it. I think it is pretty obvious how awesome this actually is.”
And I agree – this is self-evidently useful. I like “automatically fold all imports”, which is now the default behavior in a lot of IDEs. I don’t necessarily think this mode should be the default, but if it were at least an OPTION that would be fantastic.
The problem with passkeys is that they’re essentially a halfway house to a password manager
This is utter nonsense. The author has completely failed to understand the purpose of passkeys.
The entire point of passkeys is to get rid of passwords altogether. Password managers are a halfway house to secure authentication, and the fact that their password managers doesn’t support passkeys is a problem with their manager not knowing passkeys
But can a password manager know passkeys with the same usability as with passwords?
What I can do today, if I need to login into something from my wife’s computer, is to pop open bitwarden’s tab, enter my master password and copy-paste the password for the specific thing I need.
My understanding is that this is not really possible with passkeys: even if I store a passkey in Bitwarden, I can’t use it in an ad-hoc manner, I need to at least install a browser extension? Or is there some user-accessible flow where I can manually copy-paste the challenge and the response?
Of course, this whole thing with entering my master password into computer I do not own is icky security wise, but security is just the second most important feature of the password manager. The first one is me getting access to my stuff when I need it.
What I can do today, if I need to login into something from my wife’s computer, is to pop open bitwarden’s tab, enter my master password and copy-paste the password for the specific thing I need.
Yes, this is the attack vector that passkeys mitigate. Because you can, and people do, put this into a form controlled by an attacker. Passkeys make this impossible.
My understanding is that this is not really possible with passkeys: even if I store a passkey in Bitwarden, I can’t use it in an ad-hoc manner, I need to at least install a browser extension?
Either the browser builtin mechanism or a browser extension. I think Mac/iOS know directly which app is providing passkey support if not the OS internal.
Or is there some user-accessible flow where I can manually copy-paste the challenge and the response?
No, because copy-pasting the challenge-response is literally the attack vector being prevented.
Anything you copy - from the site, your phone, or whatever - can be provided by the attacker, so if you could take that and get your passkey implementation to sign a response to that and paste that back you have provided a mechanism for an attacker to authenticate themselves as you.
The entire point of the passkey process is to prevent that attack, the authentication process is basically:
Browser makes a tls connection to a host
The host provides a random token for your connection
The browser looks up the passkey it has for the host
If it finds one, it (essentially) signs the bytes from (2)
It sends that back
(1+3) mean that passkeys can’t be bypassed by a phishing site (phishing breaks autofill, but not manual fill like you’re suggesting)
(2) means that the attacker cannot control the data you’re signing
(4) means that the attacker cannot get your secret material
(5) the host can now confirm that the device performing the authentication is the device it is communicating with
the moment the you, the user, are copying content you lose all of this and you’ve reduced the security back to the level of passwords.
I found the very end of @matklad’s comment significant.
security is just the second most important feature of the password manager. The first one is me getting access to my stuff when I need it.
When push comes to shove, essentially everyone prioritises, let’s call them functional requirements, over security—and the tiny number of people who do not can be disregarded because they threw their computers into lava many years ago.
I think it’s fine for people to throw their computers into the lava. It’s not so great to roll out functionality to throw other people’s computers into the lava because you have a different opinion about security and the privilege to be able to roll your opinion out globally.
Most people just want to press a button to log in, which is what passkeys allow, and do so more securely and robustly that passwords or password managers.
Yes it’s possible that for you the trade off for security is different than for other users, but saying “passkeys are bad because my trade offs permit reduced security properties in exchange for ease of new device logins” is not a reasonable position either.
To my knowledge no one is being forced to use passkeys, you’re perfectly able to use password+2fa wherever you would like, and other people can use passkeys - mostly I’m guessing due to the more streamlined login, but for me the strong security guarantees are the compelling part.
mostly I’m guessing due to the more streamlined login
People are onboarded by what looks like a more convenient login but I (and they probably too) have no idea about the edge cases. What happens if they lose a device? No way to figure this out really without going through it.
Yes it’s possible that for you the trade off for security is different than for other users
Most people with serious security requirements seem to be using Yubikeys anyway. For most normal people passwords written in a paper book are the best solution, and critical digital logins backed by some national identity or notary scheme so they can retrieve their accounts in the case of loss.
(I just got locked out of an old Google account because I don’t have access to the 2FA phone number anymore. Let’s just say I’m glad there wasn’t really anything of value in there.)
(I just got locked out of an old Google account because I don’t have access to the 2FA phone number anymore. Let’s just say I’m glad there wasn’t really anything of value in there.)
That’s what the backup codes that Google prompts you to save in a secure place after setting up 2FA are for.
It of course perfectly illustrates the failure modes of these systems and if I a CS graduate am bit by this, then for sure most normal people do not stand a chance.
It’s too bad every large website implementing passkeys has figured out that you still have to support email-based self-service recovery to avoid a flood of support requests.
All of these protections are pointless if every passkey can be bypassed through email. Attackers have no problem walking users through recovery flows.
Suppose I have a site that I usually log into from my phone, but today I want to connect to it from a browser on my mother’s computer. Is that simply prohibited in the name of better security? Or is there a way I can accomplish it if there is no password, only a passkey.
That’s up to the site, you’re fundamentally in a position of doing a new device log in, so you’d presumably need a full 2fa dance, which they could permit via your signed in phone, or whatever other mechanism they choose. However they would still have to take steps to mitigate the risk of that being used as a phishing/social engineering attack (you basically start to run the risk of creating the kind “account recovery” style mechanism scammers use to compromise accounts).
It seems to me that there is an irreconcilable issue here: either a site allows some kind of account recovery (and “log in from my mother’s computer” is an example of “account recovery”), in which case there is a risk of it being used in a social engineering attack, or it does NOT allow any kind of account recovery, in which case accounts can’t be recovered.
What I would like to see is some enhanced forms of validation needed when doing “account recovery”. But that ends up meaning that I CANNOT “get rid of passwords and just use passkeys instead” (which is the promise) unless I also give up completely on having ways to recover an account.
Because it’s not just technically illiterate users? There are many attack vectors that are not trivial to identify and human brains are notoriously bad at reading/seeing what they expect to see. Yes of course there are people who are more gullible than others, but the security of your account should depend on more than just never ever making a mistake.
A better question there would be “why should be able to use your service securely be dependent on expert level understanding of attacks?”
If as you say the problem is just not being technically literate I’m curious if you avoid/ignore 2FA mechanisms, because those exist simply to try and provide some of the guarantees you get from passkeys. If it’s just a matter of technical literacy then it’s just an annoyance during log in.
i’m struggling to see how transferable passkeys aren’t just isomorphic to passwords in a password manager.
i suppose with certain passkey schemes you could perform a challenge-response authentication over an unencrypted channel… but i’m not sure that’s desirable tbh.
i’m struggling to see how transferable passkeys aren’t just isomorphic to passwords in a password manager.
Any “move from service A to service B” is going to involve a cryptographic exchange, e.g. service A would re-wrap the key material to a key provided by service B (this is the “don’t leak the unwrapped key material”). But I agree, my concern is that supporting that opens a vector for scammers to convince people to transfer their keys to an attacker controller application which obviously is not going to care too much about actually keeping the data secret, which is a significant reduction in the security against social engineering (in that it goes from being actually impossible to being at all possible).
It is still important to understand though that even with that obnoxious problem passkeys are still fundamentally different from passwords, the reason they’re set up the way they are is to prevent any kind of man in the middle or social engineering attack. They’re not just an HSM backed password manager (that’s a bare minimum requirement of any password manager), they’re cryptographic keys that allow authentication that also proves to a server that the client performing the authentication is the client it is talking to. That’s not something any password or password+2fa/totp authentication can do, and that’s a weakness that is constantly being exploited.
But I agree, my concern is that supporting that opens a vector for scammers to convince people to transfer their keys to an attacker controller application
This is a legitimate concern, but I think the UI challenges are solvable. First, unlike copying and pasting a password, this is not going to be a common flow. People who are happy with a single vendor’s ecosystem will never use it and the flows between the big vendors are likely to be polished.
Even without multi-vendor support, the Apple flow is pretty good. You need to log into the new device with your Apple ID. This requires a password for bootstrapping but then requires a second factor p, which is typically another Apple device that tells you the device that you’re trying to log into and where it is. The latter is the biggest UI flaw because Apple’s geolocation is terrible and seems to be no more accurate than getting the correct country, but that should be fixable. You could also add a distance-bounding protocol over Bluetooth in here for the location-looks-wrong case.
This is a flow that users will do once when setting up a new device and can be flagged with ‘this is a new device setup thing, make sure that the target is your device, never let anyone else do this’ warning.
The bigger worry for me is coercion. If you have a person and their phone, forcing them to sync their passkeys to your device is quite easy. WebAuthn doesn’t seem to have a good revocation story for ‘this copy of my keys is a stolen one, please undo everything it’s done and lock it out’. Windows Hello has device-bound credentials, which carry the device identity with them so you can tell the difference between logins by the same user from different devices, but I believe that’s a non-standard extension.
Any “move from service A to service B” is going to involve a cryptographic exchange, e.g. service A would re-wrap the key material to a key provided by service B (this is the “don’t leak the unwrapped key material”).
isn’t this already handled by oauth? or am i misunderstanding what you mean here by “service”?
It is still important to understand though that even with that obnoxious problem passkeys are still fundamentally different from passwords, the reason they’re set up the way they are is to prevent any kind of man in the middle or social engineering attack.
MITM is addressed by TLS encryption
social engineering is an entire category of attacks, changing one technical detail may make some social engineering attacks more difficult, but it most certainly does not make all social engineering attacks impossible.
i’m really just struggling to see what “the point” is. it basically seem like making it slightly easier to follow good security practices at the cost of making it significantly harder to actually use services.
isn’t this already handled by oauth? or am i misunderstanding what you mean here by “service”?
Yes, but I think entirely due to bad explanation on my part. It’s a skill :D
What I’m talking about here is the passkey provider, e.g apple’s keychain, googles , 1password, etc
Something people constantly complained about with passkeys was you create your passkeys with Service A (say keychain) and you can’t simply transfer that passkey to a different tool like 1password, what you had to do was create a new passkey with the new provider you wanted to use instead. For a couple of sites that’s not an issue, but if you were say just planning to transition to a new ecosystem and wanted to move all of your credentials over that’s a lot of work.
The thing is that the reason passkeys did not permit the extraction/transfer of the material is because scammers are exceptionally good at convincing their victims to do things, like telling the person on the phone what their 2fa token is, or installing software and giving that software excitingly elevated permissions.
Introducing the ability to export passkeys to another provider opens the previously non existence option of an attacker convincing their victim to install a piece of software and then export their passkeys to it, and in doing so provide all the material to the scammer.
It is still important to understand though that even with that obnoxious problem passkeys are still fundamentally different from passwords, the reason they’re set up the way they are is to prevent any kind of man in the middle or social engineering attack.
MITM is addressed by TLS encryption
No TLS does not resolve this kind of man in the middle, as this isn’t just about MiTM of a network connection, it MiTM of the authentication path. The user is convinced, by whatever mechanism, to open a scam page. The url is not the real website, but it is objectively true that users are bad at recognizing this. The scammers are forwarding the content of the real web page to the user of the fake url, the user enters there password in the fake page (sites are good enough at breaking password autofill that even users who are using password managers routinely fall for this), the scam page sends that back to attacker who uses that to log in. If the site has some form of 2FA, the scammer simply forwards that request to the victim, the user is expecting this, so still does not question the need and enters the code, which is forwarded to the scammer, who then has successfully compromised the account.
Passkeys tie the authentication to the actual identity of page being loaded on the authenticating machine, so this kind of MiTM simply does not work.
social engineering is an entire category of attacks, changing one technical detail may make some social engineering attacks more difficult, but it most certainly does not make all social engineering attacks impossible.
All the social engineering attacks targeting authentication revolve around getting the victim to provide their passwords and necessary 2FA tokens to the attacker, which only works because the passwords and 2fa tokens are not tied to the device or connection being authenticated.
These are the ways that scammers compromise accounts in the real world, and passkeys prevent them in a way that is fundamentally not possible via any other mechanism (in the sense of “this authentication model is the only way to prevent these kinds of attacks”, “passkeys” just happen to be the implementation+branding we have for the model).
sites are good enough at breaking password autofill that even users who are using password managers routinely fall for this
Ah, this addresses my wondering why commenters are saying password managers are so insufficient. (I hadn’t personally encountered broken password autofill not being a reasonably reliable signal that one’s not talking to the right website.)
it’s super annoying, especially now so many sites have started breaking sign in pages into multiple steps, something that serves literally no purpose other than breaking automatic sign in and encouraging users to expect random page loads and page changes during sign in. Le sigh.
Still at least there’s fairly standard mechanisms for marking a password field so the autofill is actually possible, as opposed to the relentlessly stupid and brain dead password rules and restrictions that sites keep inventing, despite decades of research showing the outcome is worse passwords that are reused, and systematically selected. Mercifully NIST has finally updated their docs to say “stupid ass rules that prevent random passwords are stupid and you suck” :D
many sites have started breaking sign in pages into multiple steps, something that serves literally no purpose other than breaking automatic sign in
In my limited experience of websites with multi-stage sign-in pages, the Google Chrome password manager copes with them, even if it needs to have the user choose from the saved username/password pairs (if there are >1) at the username step and then again at the password step, which still keeps the passwords that the user is prompted to use limited to ones associated with the domain. I don’t doubt that there are worse sign-in pages with which I’m not familiar, though!
Oh yeah, I think they all manage it now, it’s just more work for the implementation (the site navigation and UI changes makes it much harder to link account and password fields). For autofill you have to approve the autofill multiple times, etc.
It literally just makes site login slower, password autofill more annoying to implement, and trains users to expect jank and weird page loads and refreshes scattered throughout a legitimate login.
It’s just a very weird and obnoxious decision by these sites.
But if you want to be able to share keys around and back them up without some third-party service (which may be niche requirements, but they are critical to me) then you need some form of serialized secret which is effectively a password. At that point you may make it user-readable and easy to type on a keyboard. Sure, I almost never do type it. But when I do need to it is invaluable.
At that point you may make it user-readable and easy to type on a keyboard
Passkeys are not passwords that are being kept secret from you, they’re a cryptographic challenge/response process.
They’re not human typable because passkey authentication means signing a server provided nonce as proof of identity, and the server is able to verify that the machine doing the authentication is yours because an attacker is not able to get your machine to sign the data that the server has given them. I tried to summarize in my response here https://lobste.rs/s/2mgwsz/passwords_have_problems_passkeys_have#c_mp7x0o
I should have been more clear that I am talking about the storage format not the authentication protocol.
You can also do challenge response with passwords (ex: SRP, OPAQUE). My opinion is that upgrading the authentication protocol without moving away from passwords as a secret format would have been a better approach. Password managers can become more aggressive about warning users when displaying and exporting passwords (to prevent phishing and similar) but we don’t throw away the concept of passwords that is so simple to understand that my grandparents can back up their password manager into a notebook.
Basically I feel that we gave up so much for an improvement that could have been added to the simple system. With a PAKE and some UX you get basically all of the benefits of passkeys but users can still understand how it works to a degree that lets them do useful things like authenticate on a friend’s machine.
useful things like authenticate on a friend’s machine.
This seems like the core disagreement here: some view this as an important UX feature, and some consider this to be a security bug in need of prevention.
Both sides have their point! And it indeed seems that it’s not actually possible to automatically separate useful “your friend’s machine” from malicious fishing.
I wish someone has stated the tradeoff more explicitly here. It is clear what attacks are prevented. It is not completely clear which valid use-cases are prevented as a collateral damage.
The problem is less a disagreement but rather one side (the passkey side) trying to push their opinion onto the other side. I’m okay to have the option to use passkeys in addition to passwords. The passkey folks don’t seem to like it, they need to replace passwords.
I mean, in this very thread there are some comments that say that passkeys have no advantage over passwords in a password manager, and the article itself kinda-sorta claims the same.
Things are very rarely one-sided, even if it looks like that from a particular side!
How does PAKE address the benefits of pass keys? I don’t think it does at all. You can MITM PAKE just as easily as any password, it’s just that the password doesn’t get leaked - the signed response does.
PAKE is great because the underlying secret never gets conveyed but it doesn’t address phishing. PAKE’s main benefit is that you don’t end up with “mypassword1234” scattered in your nginx logs or whatever.
Also, PAKE is not straightforward to implement on the server. I know that passkeys aren’t either, but I do just want to point out that it’s not simple. User signup becomes much trickier.
Ok, in that case the issue just becomes the non-escaping nature of (the original) passkey model.
The reason for that mechanism is specifically to prevent scammers from being able to get people to provide the actual secret, because once you provide the secret to the attacker they can authenticate with the material the server provides their own connection.
The reason folk want a password, is because they want to be able access the secret, and in other words defeat the above.
I guess a good way to think about this is, how would it impact your opinion if the passwords from your password manager were just base64 encoded ECC keys?
I believe that you can only protect people from themselves so much. This is why I would rather make it a UX issue than add technical limitations and proprietary sync protocols. We can make the UX very aggressive about letting people know that they are doing something risky. (Or for corporate secrets actually block exporting).
were just base64 encoded ECC keys
If they were of a reasonable length then I would be fine with it. Honestly if a portable and reasonably human-manageable export format for passkeys was standardized would be pretty fine with the whole system. Maybe it will land some day, but I feel that this is a critical feature that should have been mandated from day one.
I believe that you can only protect people from themselves so much.
If there is a well understood, and effective, attack that works consistently, and you can simply make that attack not function why would you not?
were just base64 encoded ECC keys
If they were of a reasonable length then I would be fine with it.
right, so your issue is not the authentication scheme, which is superior to password based auth, its that you can’t simply extract the key material and use it directly yourself. Despite not being able to do any of the authentication yourself - it’s not an operation any person is doing in their head - and it means you’ve taken on the burden of verifying the host identity of the forms you’re reading and filling out (which is not necessarily the same as “what is the url the location bar shows”)
It’s at the expense of friction, not ability. You can still log in. If my phone is my passkey I can have my phone authorize me via, for example, a QR code.
Because there are downsides that are not worth it.
We can lock everyone in their own house to prevent murder, STDs and many other things. But it is not worth the tradeoff. We can force people to eat a regimented diet to prevent a huge number of illnesses. But that isn’t a world I want to live in. We should help people to the right thing, but at some point we need to give people freedom.
Really, it’s fine for me if the passwords from my password manager are just base64 encoded ECC keys. I just want to be able to use my choice of password manager, and sync its encrypted database file however I please. I’m not hugely concerned if there is a standard way for the browser and the password manager to communicate that’s not copy-and-paste, either. What I am concerned about is having my choice of password manager constrained to e.g. Apple or Google.
What I am concerned about is having my choice of password manager constrained to e.g. Apple or Google.
It isn‘t with Passkeys. 1Password implements them, and any other password manager can too. Apple, and I guess Google and Microsoft also, provide APIs to hook into both password autofill and passkeys for 3rd party managers to use.
This release broke Datasette due to one of my dependencies not working with Python 3.13 yet - I pushed an update just now that fixes that, details of the change in this issue.
The dataclass inheritance hierarchy is supposed to require all classes to be either frozen or non frozen, this works properly for checking that an unfrozen class does not inherit from any frozen classes, but it allows frozen classes to inherit from unfrozen ones as long as there’s at least one frozen class in the MI
Unfortunately my Pint dependency was relying on that bug.
FYI: Python does not use SemVer and never really has. There’s always been a rolling deprecation cycle in Python releases, and it’s always been important to pay attention to deprecation warnings and release notes.
As to why backwards-incompatible changes get made at all, it’s because the alternative is to be something like Java, where there are decades of accumulated “no, don’t use that API for this, or that one, or that one, or that one” footguns because an absolute obedience to never making a breaking change prevents the removal of known-bad APIs.
“no, don’t use that API for this, or that one, or that one, or that one” footguns because an absolute obedience to never making a breaking change prevents the removal of known-bad APIs.
Doesn’t Python have a bunch of known crappy modules in its standard library?
Python’s been working on removing a bunch of outdated modules/APIs in its standard library and replacing them with or directing people to better alternatives.
Of course, doing so breaks backward compatibility, which people then complain about.
I think there’s a difference between “this outright doesn’t work” (which I think is mainly a thing in something like datetime.replace just doing the ‘wrong’ thing when you pass in a timezone for example), and “this is not a nice API”, which I think is how I would qualify some standard library APIs.
And even then, honestly? The standard library is pretty fine in my experience. The main complaint I’d have is urllib stuff just not really working out of the box on Mac without shenanigans to get certs working first.
In my experience, everyone gets dates and time wrong, including me. The implicit assumptions list in date and time math is 400 miles long when printed in size 1 font..
I’ve never looked at whenever, but to believe it fully encompasses the date and time problem is probably ridiculous. When countries can’t even agree on what the right date or time is at a given place, and cultures/groups of people sometimes have their own date and time interpretation separate from their countries, there is zero hope anyone can get it right in a library.
Fair enough. I have the luck of both living in places without DST and also knowing to never use naive datetimes, so most of these are non-issues for me. But it would be nice for all of this stuff to just be…. better.
datetimes are total footguns in general if you just open up the standard library and use it.
I’m happy at least tzinfo is easier to get a hold of than it used to.
I think my favorite bit is from footnote 35:
I suspect his corrected version still has a problem.
His first issue was that the next day’s directory might be created early… giving a window on the order of ~5 minutes in which it could erroneously be processed and marked complete. The fix was to mark only previous day directories as completed.
I would like to point out that there is also a race condition around the precise moment when the date changes. We know it takes on the order of ~1 minute to read a directory, so depending on how the code is written there might be a ~1 minute window or perhaps only a ~few milliseconds window in which this application and the other applications that are writing records might disagree about what the date is. Either way, it leaves a possibility for imperfect data.
Given that skipping historical directories is an optimization (and the system worked without that optimization in place), I would probably switch to marking directories as “completely scanned” only if they are older than yesterday (instead of older than today). That means we will continue re-scanning yesterday’s directory for a full day, but it also gives us a full 24 hours to be confident that all systems agree the date has changed. (If the optimization were more important, I might give it 30 mins instead of 24 hours – but certainly something much larger than the expected run time of any processes that probably cache the date.)
For those who didn’t read the article, here is the final line: “Maybe we need a new FAANG acronym that covers OpenAI, Anthropic, Google, Meta and Amazon. I like GAMOA.”
Maybe we shouldn’t? More often than not, naming things is what create them, like FAANG and BRICS.
I’d prefer OMAGA.
Why exclude Microsoft, who are essentially bankrolling OpenAI and is the company most visibly offering GenAI to consumers and businesses?
Because they haven’t produced their own GPT-4 class model yet. I like the Phi series but they’re not at the same level as GPT-4o/Claude 3.5 Sonnet/Gemini 1.5 Pro yet.
The insights from someone who has serious experience is incredibly valuable.
But I find myself a bit perplexed. I fully understand the complaint (of Haskell) that there are many ways to solve each problem and these sometimes don’t work together or make your build chain and tooling unstable. I fully understand the complaint (of Haskell) that error messages are unhelpful.
I do not fully understand what appears to be the author’s largest complaint: that there are several libraries available to solve most problems and so a programmer is forced to choose. Why allow this to be a drag? Let someone publish their list of library preferences. Or pick the library that DOESN’T require you to recompile your IDE. Or pick the one that is first in alphabetical order. I don’t see why more choices is harmful. (Except perhaps if fragmentation of effort means that none of the libraries is actually complete.)
I’m sure there IS something behind this claim. What am I missing here?
In my personal experience, choosing a library for an industrial project comes with the implicit responsibility of choosing the “best” one—or, at least, not a clearly inferior one—per some calculus of “best.” Further, the decision process might go into an ADR or similar documentation, just depending. These qualifying factors include things like assessing the amount of the library’s API churn, or the primary library’s author history of abandoning projects. How long might this library be maintained (think Lindy effect)? Is there a big semver major bump coming up for the library such that its documentation is dithered and you’ll also need to go water it in a month? Is its interface layer ergonomic for your anticipated use cases? Do the library’s motivations make sense, and are they aligned with your goals (ex. performance, simplicity, API alignment with a framework that you’re using/not-using, etc.)? Yes, I understand that libraries should be a black box, but if you think you’ll potentially reach or exceed the boundaries of the library’s capabilities, then you might want to assess the readability of its code (and/or perhaps check its test suite). Is its documentation robust, yet also commensurate, length- and tone-wise, with the library’s impact or importance for your project? Has the primary corporate sponsor changed a project’s license for a similar library in the past? What are the general vibes of the main authors/maintainers (ex. ghost, dilettante, star-grabber, well-known serious industry veteran, algorithm wonk, etc.)? Then there’s baseline stuff like licensing and “is it performant?”.
So I would say if I’m faced with one single JSON library that I need in an ecosystem that seems roughly right, I would circle the car real fast, kick the tires, grab the keys and drive it off the lot. If I’m in a different ecosystem and faced with 10 JSON libraries, then I know that I’m going to have spend time doing at least some of the vetting mentioned above.
Less serious projects have lower stakes, so calibrate effort accordingly.
The drag I found was that there’d be a handful of incompatible libraries defining some core data structures within a field, and the rest of the libraries would be pretty evenly split on which one they depend on. Then you end up doing manual dependency resolution, rewriting things a dozen times out of indecision, reimplementing functionality you know already exists but can’t use, trying some horrible lossy conversion between the competing ecosystems, and so on.
IIRC I found this in audio, MIDI, 2D/3D graphics, and probably more.
Oh! That seems to me to be a much stronger case than the other reasoning I have heard. It’s not arguing that I need to expend time worrying about which library to use (which could be eliminated by simply picking one and foregoing the benefits of all the others). This illustrates that multiple libraries can be strictly inferior to having a single library, because it causes splits in OTHER libraries that depend on it.
This is a falsehood some people in the intersection of the data oriented design and C crowd love to sell. RAII works fine with batches, it’s just the RAII object is the batch instead of the elements inside. Even if the individual elements have destructors, if you have an alternative implementation for batches C++ has all the tools to avoid the automatic destructor calls, just placement new into a char buffer and then you can run whatever batch logic instead. Don’t try bringing this up with any of them though or you’ll get an instant block.
I do highly performance sensitive latency work with tighter deadlines than gamedev and still use destructors for all the long lived object cleanup. For objects that are churned aggressively avoiding destructor calls isn’t any different than avoiding any other method call.
Agreed, this post is making some wild claims that don’t hold up in my experience. I’m writing a high-performance compiler in Rust, and most state exists as plain-old datatypes in re-usable memory arenas that are freed at the end of execution. RAII is not involved in the hot phase of the compiler. Neither are any smart pointers or linked lists.
I simply find the argument unconvincing. Visual Studio has performance problems related to destructors => RAII causes slow software?
Agreed. I like Zig and appreciate Loris’ work, but I don’t understand this argument as well.
“Exists in tension” seems accurate to me. Yes, you can do batches with RAII, but in practice RAII languages lead to ecosystems and conventions that make it difficult. The majority of Rust crates use standard library containers and provide no fine grained control over their allocation. You could imagine a Rust where allocators were always passed around, but RAII would still constrain things because batching to change deallocation patterns would require changing types. I think the flexibility (and pitfalls) of Zig’s comptime duck typing vs. Rust traits is sort of analogous to the situation with no RAII vs. RAII.
I think it’s the case that library interfaces tend not to hand control of allocations to the caller but I think that’s because there’s almost never pressure to do so. When I’ve wanted this I’ve just forked or submitted patches to allow me to do so and it’s been pretty trivial.
Similarly, most libraries that use a HashMap do not expose a way to pick the hash algorithm. This is a bummer because I expect the use of siphash to cause way more performance problems than deallocations. And so I just submit PRs.
Yes. I write Zig every day, and yet it feels like a big miss, and, idk, populist? “But don’t just take my word for it.” Feels like too much trying to do ‘convincing’ as opposed to elucidating something neat. (But I guess this is kind of the entire sphere it’s written in; what does the “Rust/Linux Drama” need? Clearly, another contender!)
To be fair, invalidating this specific argument against RAII does not invalidate the entire post.
You write Zig every day? What kind of program are you working on?
It doesn’t, but without it I don’t really see the post offering anything other than contention for the sake of marketing.
I spend somewhere between 2 to 8 hours a day working on my own projects. (“2” on days I also do paid work, but that’s only two days a week.) Zig has been my language of choice for four or five years now; you can see a list on my GitHub profile. A lot of my recent work with it is private.
Impressive commitment to Zig! Thanks for sharing.
Thank you! I really like it, and I’m a little sad that Rust — which I still use often, maintain FOSS software in, and advocate for happily! — has narrowed the conversation around lower-level general-purpose programming languages in a direction where many now reject out of hand anything without language-enforced memory safety. It’s a really nice thing to have, and Rust is often a great choice, but I don’t love how dogmatic the discourse can be at the expense of new ideas and ways of thinking.
I very much agree. A Zig program written in a data-oriented programming style, where most objects are referenced using indices into large arrays (potentially associated to a generation number) should be mostly memory safe. But I haven’t written enough Zig to confirm this intuition.
I don’t remember the arguments against RAII much (has been a few years since) but that Zig doesn’t have RAII feels like an odd omission given the rest of the language design. It’s somewhat puzzling to me.
Hm, it’s pretty clear RAII goes against the design of Zig. It could be argued that it’d be a good tradeoff still, but it definitely goes against the grain.
__deinit__function if available” would the the sole place where that sort of thing would be happeningdefer fits Zig very well, RAII not at all.
I was unaware that Zig discourages holding on to the allocator. I did not spend enough time with Zig but for instance if you have an
ArrayListyou can defer.deinit()and it will work just fine. So I was assuming that this pattern:Could be turned into something more implicit like
I understand that “hidden control flow” is something that zig advertises itself against, but at the end of the day
deferis already something that makes this slightly harder to understand. I do understand that this is something that the language opted against but it still feels odd to me that no real attempt was made (seemingly?) to avoiddefer.But it very much sounds like that this pattern is on the way out anyways.
Zig’s
std.HashMapfamily stores a per-collection allocator inside the struct that is passed in exactly once through theinitmethod. Idk how that can be considered non-idiomatic if it’s part of the standard library.It is getting removed! https://github.com/ziglang/zig/pull/22087
Zig is a pre 1.0 language. Code in stdlib is not necessary idiomatic both because there’s still idiom churn, and because it was not uniformly audited for code quality.
As someone who doesn’t use Zig or follow it closely, both the fact that that change is being made and the reason behind it are really interesting. Thanks for sharing it here
You might also like https://matklad.github.io/2020/12/28/csdi.html then, as a generalization of what’s happening with Zig collections.
That’s an interesting development. Thanks for informing me!
It would completely change the design of the language and its approach to memory and resource management.
I’ve never used placement
new, so I don’t know about that, so my question is, how do you do that? Take for instance a simple case where I need a destructor:If I have a bunch of elements that are both constructed at the same time, then later destroyed at the same time, I can imagine having a dedicated
Element_listclass for this, but never having used placementnew, I don’t know right now how I would batch the allocations and deallocations.And what if my elements are constructed at different times, but then later destroyed at the same time? How could we make that work?
I think I have an idea about their perspective. I’ve never done Rust, but I do have about 15 years of C++ experience. Not once in my career have I seen a placement
new. Not in my own code, not in my colleagues’ code, not in any code I have ever looked at. I know it’s a thing when someone mentions it, but that’s about it. As far as I am concerned it’s just one of the many obscure corners of C++. Now imagine you go to someone like me, and tell them to “just placement new” like it’s a beginner technique everyone ought to have learned in their first year of C++.I don’t expect this to go down very well, especially if you start calling out skill issues explicitly.
I’m a little bit surprised, because I’ve had the opposite experience. Systems programming in C++ uses placement new all of the time, because it’s the way that you integrate with custom allocators.
In C++, there are four steps to creating and destroying an object:
When you use the default
newordeleteoperators, you’re doing two of these: first calling the globalnew, which returns a pointer to some memory (or throws an exception if allocation fails) and then calling the constructor, then calling the destructor. Bothnewanddeleteare simply operators that can be overloaded, so you can provide your own, either globally, globally for some overload, or per class.Placement new has weird syntax, but is conceptually simple. When you do
new SomeClass(...), you’re actually writingnew ({arguments to new}) SomeClass({arguments to SomeClass's constructor}). You can overloadnewbased on the types of the arguments passed to it. Placement new is a special variant that takes avoid*and doesn’t do anything (it’s the identity function). When you donew (somePointer) SomeClass(Args...), wheresomePointeris an existing allocation, the placement new simply returnssomePointer. It’s up to you to ensure that you have space here.If you want to allocate memory with
mallocin C++ and construct an object in it, you’d write something like this (not exactly like this, because this will leak memory if the constructor throws):This separates the allocation and construction: you’re calling
mallocto allocate the object and then calling placement new to call the constructor and change the type of the underlying memory toT.Similarly, you can separate the destruction and deallocation like this (same exception-safety warning applies):
In your example,
std::unique_ptrhas a destructor that callsdelete. This may be the global delete, or it may be somedeleteprovided byFoo,Bar, orBaz.If you’re doing placement new, you can still use
std::unique_ptr, but you must pass a custom deleter. This can call the destructor but not reclaim the memory. For example, you could allocate space for all three of the objects in your ‘object’ with a single allocation and use a custom deleter that didn’t free the memory instd::unique_ptr.Most of the standard collection types take an allocator as a template argument, which makes it possible to abstract over these things, in theory (in practice, the allocator APIs are not well designed).
LLVM does arena allocation by providing making some classes constructors private and exposing them with factory methods on the object that owns the memory. This does bump allocation and then does placement new. You just ‘leak’ the objects created this way, they’re collected when the parent object is destroyed.
Thanks for the explanation, that helps a ton.
I’ve done very little systems programming in C++. Almost all the C++ code I have worked with was application code, and even the “system” portion hardly did any system call. Also, most C++ programs I’ve worked with would have been better of using a garbage collected language, but that wasn’t my choice.
This may explain the differences in our experiences.
Yup, that’s a very different experience. Most C++ application code I’ve seen would be better in Java, Objective-C, C#, or one of a dozen other languages. It’s a good systems language, it’s a mediocre application language.
For use in a kernel, or writing a memory allocator, GC, or language runtime, C++ is pretty nice. It’s far better than C and I think the tradeoff relative to Rust is complicated. For writing applications, it’s just about usable but very rarely the best choice. Most of the time I use C++ in userspace, I use it because Sol3 lets me easily expose things to Lua.
I think it very much also depends on the subset of C++ you’re working with, at a former job I worked on a server application that might have worked in Java with some pains (interfacing with C libs quite a bit), and in (2020?) or later it should have probably be done in Rust but it was just slightly older that Rust had gained… traction or 1.0 release. It was (or still is, probably) written in the most high-level Java-like C++ I’ve ever seen due to extensive use of Qt and smart pointers. I’m not saying we never had segfaults or memory problems but not nearly as many as I would have expected. But yeah, I think I’ve never even heard about this placement new thing (reading up now), but I’m also not calling myself a C++ programmer.
Placement new is half the story, you also need to be aware that you can invoke destructors explicitly.
A trivial example looks like
If you want to defer the construction of multiple foos but have a single allocation you can imagine
char foos_storage[sizeof(foo)*10]and looping to call the destructors. Of course you can heap allocate the storage too.However, you mostly don’t do this because if you looking for something that keeps a list of elements and uses placement new to batch allocation/deallocation that’s just
std::vector<element>.Likewise if I wanted to batch the allocation of Foo Bar and Baz in Element I probably would just make them normal members.
Each element and its members is now a single allocation and you can stick a bunch of them in a vector for more batching.
If you want to defer the initialization of the members but not the allocation you can use
std::optionalto not need to deal with the nitty gritty of placement new and explicitly calling the destructor.IME placement new comes up implementing containers and basically not much otherwise.
Note that since C++20+ you should rather use std::construct_at and std::destroy_at since these don’t require spelling the type and can be used inside constexpr contexts.
You likely use placement new every day indirectly without realizing it, it’s used by std::vector and other container implementations.
When you write
new T(arg)two things happen, the memory is allocated and the constructor runs. All placement new does is let you skip the memory allocation and instead run the constructor on memory you provide. The syntax is a little weirdnew(pointer) T(arg). But that’s it! That will create aTat the address stored inpointer, and it will return aT*pointing to the same address (but it will be aT*whereaspointerwas probablyvoid*orchar*). Without this technique, you can’t implement std::vector, because you need to be able to allocate room for an array of T without constructing the T right away since there’s a difference between size and capacity. Later to destroy the item you do the reverse, you call the destructor manuallyfoo->~T(), then deallocate the memory. When you clear a vector it runs the destructors one by one but then gives the memory back all at once with a single free/delete. If you had a type that you wanted to be able to do a sort of batch destruction on (maybe the destructor does some work that you can SIMD’ify), you’d need to make your own function and call it with the array instead of the individual destructors, then free the memory as normal.I’m not trying to call anybody out for having a skill issue, but I am calling out people who are saying it’s necessary to abandon the language to deal with one pattern without actually knowing what facilities the language provides.
What would this look like in practice? How do you avoid shooting yourself in the foot due to a custom destructor? Is there a known pattern here?
There are different ways you could do it but one way would be to have a template that you specialize for arrays of T, where the default implementation does one by one destruction and the specialization does the batch version. You could also override regular operator delete to not have an implementation to force people to remember to use a special function.
So, how are people supposed to prove, over the Internet, that they’re blind, and not pretending to be blind for fraudulent purposes? And/or: How can the other side accurately determine blindness across the Internet?
If neither of those things can be done at any reasonable rate of success, what is to do be done? It seems like a hard problem to solve.
They should not be expected to prove that. Instead, websites should offer accessibility to the blind where possible (through things like screen readers) for any user who wants it, not just for blind users.
Maybe I just had eye surgery and I’m not supposed to view screens for the next 2 weeks. Maybe I just feel like lying in bed with my eyes closed and browsing your website. Why should you care?
I’ve been saying it for a long time that accessibility is for everyone, not only for people who can’t use the UI without it at all. Screen magnification and high-contrast themes are essential for people with serious vision impairments, but they are still great if one’s eyes are merely tired.
I have a friend who uses magnification in the browser and the terminal emulator as an alternative to wearing glasses all the time. Would anyone dare to say that one needs to prove that they are legally blind and already wear the strongest optically feasible glasses before they can also use screen magnification?
People shouldn’t have to prove that they’re disabled or be subjected to any form of test from the other side in order to be provided accessible access, for any product, ever. This is especially true before the person asking for access has done anything that could be construed as actual fraud (ie prematurely defensive design is offensive by design).
I don’t think that’s what they should be proving at all. We ask users who aren’t blind to do a thing that we believe demonstrates that they’re human as opposed to a computer program attempting to simulate a human’s attention.
We should still be trying to demonstrate the same thing for people whose physical differences make it impossible for them to complete our automated tests. It might be more labor intensive for the site operator to vet them in a less automated way, but the operator should be looking to establish that the user is a real human, not to establish the user’s blindness or lack thereof.
Sure it’s more manual and more costly for that kind of demonstration, but businesses are required to provide that sort of accommodation, and it’s part of the cost of preventing bots, if you’ve decided your business needs to prevent bots.
That sounds like a whole lot of not blind people’s problem?
The author of the original blog post is in fact blind. hCaptcha’s ban was groundless.
Exactly, and my point is that it’s not right for hCaptcha to make it blind people’s problems that they find it difficult to prove that someone is blind.
Ah. Xe had interpreted your original post as “Why are all the sighted people here and in the blog post complaining so much?”
If Michael Taboada (the author) were inclined to do so, he could contact the owners of websites based in USA that use hCaptcha. He could reach out to their legal team and let them know that since he, as a blind person, was banned from using their website, that he was in a position to sue them under the ADA. He could stress that he didn’t INTEND to sue them, but that he was reaching out as a favor to let them know about their liability.
Many companies would react very strongly to this. And hCaptcha might respond differently to their customers than they do to an individual who they seem to assume is trying to hack their system.
Thank you for the suggestion, I’ll let him know.
I think any compiler that wants to self-host ought to provide something like this: a chain of builds which can be used to prove that it is free of Reflections on Trusting Trust issues.
I think that the ideal way to store this chain of builds would be:
(1) A chain of build scripts, rather like what Motiejus Jakštys did here, which can be run in series to recreate the current build. The difference being that the compiler team provides the list of build scripts instead of Motiejus having to recreate it with help from Hilton Chain.
(2) The binary from each stage. Storing this enables the chain to be verified probabilistically WITHOUT needing to build the whole thing – if any link can easily be verified with a single build, then one can randomly sample a few links to confirm with reasonable confidence that if we NEVER see a failure-to-reproduce, then the whole chain must be reliable.
Would it be sufficient to store good cryptographic hashes of all these artifacts rather than the artifacts themselves, for large space savings?
Sadly, no. (I originally planned to write that we needed to store only the hashes, but then realized why it wouldn’t work.)
If we stored only the cryptographic hash at each stage then a verifier could not verify a single link in the chain… they would need the artifact from the previous stage which could only be obtained by starting from the beginning and running all the way through.
Do you think we could just use git tree hashes as our references and store all of this in the one repo? Git stores the whole Merkle chain/tree and can be pretty easily verified…
(1) I’m working to set up the polling place. I’m in the US and we vote on Tuesday. I am also responsible for running my local voting location. My innovation this year is to invite local KIDS to cast ballots for whether to hold a “campfire” or a “board game night” in ~2 weeks – then I’ll hold that event for the community.
(2) In any free time I’ll be trying to make as much progress as possible on a coding project for work. We’re establishing a new framework and I want to lay out a lot of best-practice examples before more than a few team members are working in it.
While I agree that online conferences differ from the on-site ones, I think that what’s not taken into consideration is that most of on-site conferences are not accessible to a lot of people and tend to be quite expensive. It’s fun that the author has a choice to decide which one to attend, many people don’t have that choice and still want to participate to conferences.
Doesn’t the author cover this in the last paragraph, which is about the advantage of “Affordability”?
…and also the previous point the author makes about “Accessibility” (transcription of talks can be provided since they’re available beforehand).
Under Java, David Teller lists the following:
Really? OK – that sounds interesting… just how IS finally implemented in the JVM? Does anyone know what he means here?
I believe most JVMs use a second return register that the branch on after calls to jump to catch and cleanup calls. VMKit originally used the Itanium unwind model and it was far too slow in Java where exceptions are common.
The way I’d like to implement it is to return either a value or an exception object in the return register and use the carry flag as the discriminator, so each call (to a possibly throwing function) is a call followed by a branch on carry. Most CPUs statically predict branch-on-carry as not-taken and so this is basically free (tiny increase in I-cache usage) in the no-exception case, but also fairly cheap (pipeline bubble from incorrect speculation) if an exception is thrown.
On ARM, I remember the convention was to use the overflow flag for this. e.g. http://www.riscos.com/support/developers/prm/errors.html
This is the BSD system call calling convention as well. Return values are either integer values (or, in a very small number of cases such as
mmap, pointers) orerrnovalues. If the carry flag is set, they’reerrnovalues. After the syscall / SWI instruction, you can branch on carry to the routine that sets errno.On MIPS and RISC-V, I think a second return register is used to get the same thing, but that doesn’t play as nicely with the branch predictor (and precludes exposing system calls that return two things or errno).
Linux instead requires that all system calls return negative errno values or non-negative success values.
I find there is some insight in the observation at the top that events “trigger actions” and they “carry data” and that these two different roles need to be respected.
In fact, I have some design issues right now within my organization where there is a conflict between some groups who mandated that events always contain exactly all the fields within the domain, and others who complain that (1) this means there isn’t a field telling “what happened” and (2) that makes everything slow because it must go to several different systems to collect all of that data.
I have been making the argument that the single contains-all-the-fields event was impractical; perhaps I should instead have been making the argument that there are two separate use cases, and for the “trigger action” use case we should have fewer fields.
Notice that this is DIFFERENT from what Lutz Hühnken (the author of the piece) concludes. He concludes that you should reach a happy medium: include the trigger information plus whatever “wider” data is easy. My own takeaway is you should have two events: a trigger event and a wide event. (And, of course, feel free to skip either one if it won’t actually be needed.)
This is awesome feedback, thanks! Learning that a) others have similar experiences and b) came to other (maybe better?) solutions alone was worth writing it down.
Thanks. I appreciated the article.
Not a flutter developer but shame it had to come to this. Google could have tapped into a bunch of skilled volunteer contributors. Instead PRs and bug reports are just being left to stagnate. If this gets traction hopefully it’s a wakeup call for Google.
Increasingly does seem short-sighted to trust Google with the stewardship of anything these days. I can’t guarantee they won’t randomly kill off half of Google cloud tomorrow, let alone keep a framework on life support.
Hello, Flutter contributor here!
We definitely have a long backlog of issues and feature requests. We’re the victim of our own success, we don’t have the resources to fix every single bug or add every feature. That’s one of the benefits of open-source though, if something is important to your organization, you can contribute!
However, we take PRs and contributions seriously. We have triage processes to review contributions within a week. If you have examples of PRs stagnating, please let us know. This would be unexpected and something we should fix.
Why is this buried several levels deep in the conversation?
Ultimately, either the Flutter team has the resources to review contributions within a week, or they do not have the necessary resources. If they DO have the ability to do timely reviews of contributions (and a willingness to accept properly-written contributions), then no fork is needed, although perhaps there IS a need for a group of qualified contributors outside of Google to review issue reports and feature requests and produce working contributions. If the Flutter team does NOT have the ability to do timely reviews of contributions then they either need to cooperate with someone who does have that time, OR a fork is needed.
Can we make communication about this work?
What is buried several levels deep? The comment to which you are replying is a reply to a top-level comment, so I would call it only two levels deep.
Sounds like you need to reach out to the author of the article
It’s not only happening there but everywhere in Google’s ecosystem. I completely lost faith in all their services.
Trying to capture the good practices in systems instead of relying on heroes is very sound advice.
The article goes further and says “If you have strong enough rules, you don’t need to worry about the skills of the programmers.”
I don’t think that is at all accurate, and I don’t think the article provides any support for it. An example they provide is this: “One of the solutions was to introduce a rule to memoize everything returned by hooks, to automatically prevent future problems with memoization.”
I can easily believe that most of the things returned by hooks should be memoized. I am skeptical of the claim that ALL things returned by hooks should be memoized. So a library or tool which automatically memoized the results with a way to annotate that some should be omitted might be a great idea. Hiring unskilled developers because the memoization is automatic is NOT a good idea in my experience.
One of the items is “Test all the fucking time!”. I have noticed that when I (as a veteran developer with ~30 yrs experience) sit down with a relatively new developer one of the glaring differences is that I write a few lines of code then test it; they often start by writing a page of code.
Another difference (not mentioned in this article) is that I spend a lot of time writing specifications for boundaries – things like a comment defining how a function is designed to behave, while new developers often just dive into the code first and only think about the interfaces afterward (if at all). This is a little more than Rodrigo’s “Packaging matters”.
Interesting. I definitely try to test the code I write, but I have rarely been in an environment where writing unit tests for everything is feasible. I mostly test locally whatever I can and manually test on development. In my experience this is more efficient than trying to test everything exhaustively, since testing exhaustively means I need to write mocks, use dependency injection, etc. This takes time, makes the code more complicated, and often the bugs are in the integration between services anyway: If a test using a mock service passes it only really says the code does what you expect in this particular case if the service behaves exactly the way you expect it to.
A notable exception is code that is critical and/or complex. Most of the code I work on nowadays is simple boilerplate stuff that reads/writes from multiple data sources, and writing tests for that will probably take longer than writing the code itself. Also, if you get it wrong, it’s easy to see and fix. I also worked at a compiler company, where most of the changes would fall in this category: I simply would not think of pushing a change without multiple unit tests.
In the end all “rules” for programming boil down to “use common sense”.
When I say I “test” after a few lines of code, I don’t necessarily mean unit tests. In some environments that is the right tool, in other environments “run it under the debugger” or “use the repl” might be it. I just want to something to confirm that things are behaving as I expect them to, and I want to do that as soon as I have completed a single “thought”, not after I have created a whole structure.
That rings true for me also. The simpler the the code, the more of it I test in a single bite.
The original article does, though, unless I’m misreading.
this falls pretty squarely in the category of “technically true but missing the point”.
Is Go’s panic/recover mechanism fundamentally exception-like in nature? Yes, it is. Are there codepaths that exist in which panic/recover is used for error handling? Yes, there are.
Those cases are few and far between, they’re not the dominant and main mechanism for error handling. The stdlib and the ecosystem uses the
errorinterface and multiple return for error-handling in the vast majority of error-handling codepaths, to such a degree that it’s conventional wisdom that a package should not panic and expect that a caller callrecover().It would be strictly more accurate to say “in Go, errors are nearly always signalled through values that use the
errortype, which is an interface, which is a boxed fat pointer on the heap, and represents a node in a tree oferrorvalues that is traversable using the functionality provided by theerrorspackage, except in a few situations where they usepanicandrecover, but generally speaking it’s unlikely you’ll ever encounter a library that callspanicand expects you torecover”, but that’s a lot of words, so we just say “Go doesn’t use exceptions for error handling”, because for nearly every person in nearly every situation that’s the useful level of accuracy that people need when they are learning the language and not fretting about every corner case up front.The point is when you see this code:
In Zig it is correct code. In Go it is incorrect code.
Go and the ecosystem usually don’t focus on correctness, and are fine with 90% solutions. So the fact that there’s a corner case (
bar()panics) wouldn’t be a concern.You’ll just get a crash and a new K8s pod, so don’t worry about it ;)
An attitude like that makes me concerned that I can never rely on ordinary Go code.
I think ordinary Go code doesn’t run in critical contexts, so that attitude is mostly fine in practice. You just don’t want to use Go outside its target/niche.
From a design perspective, Rob Pike infamously (basically) said Googlers are not to trust with anything advanced like generics or iterators; that’s for researchers.
And simplicity is the Go ethos, even at the cost of trading off correctness in some cases.
I personally find that mentality condescending and sad as a programmer that likes to perfect their craft, and robust code, even if just to explicitly acknowledge something’s not handled.
That being said Go is not all bad, my favorite things about it are cooperative cancellation via Context, and how tooling just works*.
I do have a lot of critical opinions about it because I’ve used it a lot and my values don’t align.
*minus the part about proxying every dependecy download through Google, and the compiler downloading and running newer versions on the fly, both showing the Google influence/values again
The net/http library uses recover by default, as do many other libraries, so you may not get a top level crash as you describe.
Yes
recover()is sprinkled at appropriate boundaries, such as request handling fornet/http, and that’s definitely a good thing.There’s also cases where you can’t
recover(): some runtime behavior is a “fatal error”, not a panic. For instance unlocking an unlocked mutex, or concurrent writes to a map.I’m not against
panicat all, it makes sense in most languages, and definitely in Go’s target use cases.That doesn’t conflict with my opinion that “usually don’t focus on correctness” is a fair assessment. The rest of my comment is maybe a bit too snarky because I was feeling jokey.
Will the pod spawn though? Cause k8s is written in golang…
Can you explain where the problem is here?
If
bar()panics, thenx.ahas been modified butx.bhasn’t.A correct form would be:
Shit I never considered that.
That’s the reason why Rust’s stdlib locks support poisoning.
TBF without effects it’s mostly an annoyance / drag.
Yeah on this, Rust added support for unpoisoning mutexes so you can fix state
In practice, there is no problem.
In theory, a panic may happen so that bar() and/or unlock() are not called.
But
unlock()is called if the program panics: https://play.golang.com/p/XiCz846Se3gSo I guess that means the problem is that the code doesn’t set
x.aback to what it was originally, as though in a database transaction, in the case thatbar()panics?If
bar()panics, in 99% of cases we should be letting the program crash so partial update of an object doesn’t really matter anyway. Go is being pragmatic here.Yeah, I think the problem arises if someone calls
recover()in thedefer, then they end up with broken state (like so).Ah, good point. You are right!
This was my thought as well. Correct me if am wrong but shouldn’t we be aware enough to handle this case when we implement it like that? Or the requirement is that the compiler should tell it?
“Go doesn’t use exceptions for error handling” is a lie, regardless of how convenient of a lie it is. A newcomer to the language is building a knowledge foundation that is wrong if presented with such statement.
Just say that there are exceptions but should almost never be used because reasons X Y Z.
Well it doesn’t use exceptions for error handling by convention. But you can, and it does have exceptions. I think the statement is still fair.
I love the audacity of writing “I will not try to motivate it. I think it is pretty obvious how awesome this actually is.”
And I agree – this is self-evidently useful. I like “automatically fold all imports”, which is now the default behavior in a lot of IDEs. I don’t necessarily think this mode should be the default, but if it were at least an OPTION that would be fantastic.
(1) I have only read the docs, not actually tried this. But it sounds quite promising.
(2) My favorite feature is the “backpressure”. Such a simple concept, but in the real world I expect it to make a big difference.
This is utter nonsense. The author has completely failed to understand the purpose of passkeys.
The entire point of passkeys is to get rid of passwords altogether. Password managers are a halfway house to secure authentication, and the fact that their password managers doesn’t support passkeys is a problem with their manager not knowing passkeys
But can a password manager know passkeys with the same usability as with passwords?
What I can do today, if I need to login into something from my wife’s computer, is to pop open bitwarden’s tab, enter my master password and copy-paste the password for the specific thing I need.
My understanding is that this is not really possible with passkeys: even if I store a passkey in Bitwarden, I can’t use it in an ad-hoc manner, I need to at least install a browser extension? Or is there some user-accessible flow where I can manually copy-paste the challenge and the response?
Of course, this whole thing with entering my master password into computer I do not own is icky security wise, but security is just the second most important feature of the password manager. The first one is me getting access to my stuff when I need it.
Yes, this is the attack vector that passkeys mitigate. Because you can, and people do, put this into a form controlled by an attacker. Passkeys make this impossible.
Either the browser builtin mechanism or a browser extension. I think Mac/iOS know directly which app is providing passkey support if not the OS internal.
No, because copy-pasting the challenge-response is literally the attack vector being prevented.
Anything you copy - from the site, your phone, or whatever - can be provided by the attacker, so if you could take that and get your passkey implementation to sign a response to that and paste that back you have provided a mechanism for an attacker to authenticate themselves as you.
The entire point of the passkey process is to prevent that attack, the authentication process is basically:
(1+3) mean that passkeys can’t be bypassed by a phishing site (phishing breaks autofill, but not manual fill like you’re suggesting)
(2) means that the attacker cannot control the data you’re signing
(4) means that the attacker cannot get your secret material
(5) the host can now confirm that the device performing the authentication is the device it is communicating with
the moment the you, the user, are copying content you lose all of this and you’ve reduced the security back to the level of passwords.
I too can make very secure systems if I ignore all the ways that people need and want to use these systems.
I found the very end of @matklad’s comment significant.
When push comes to shove, essentially everyone prioritises, let’s call them functional requirements, over security—and the tiny number of people who do not can be disregarded because they threw their computers into lava many years ago.
I think it’s fine for people to throw their computers into the lava. It’s not so great to roll out functionality to throw other people’s computers into the lava because you have a different opinion about security and the privilege to be able to roll your opinion out globally.
People forget that availability is part of security. If your only way to access a service is a passkey you can’t use, you can’t access your service.
Most people just want to press a button to log in, which is what passkeys allow, and do so more securely and robustly that passwords or password managers.
Yes it’s possible that for you the trade off for security is different than for other users, but saying “passkeys are bad because my trade offs permit reduced security properties in exchange for ease of new device logins” is not a reasonable position either.
To my knowledge no one is being forced to use passkeys, you’re perfectly able to use password+2fa wherever you would like, and other people can use passkeys - mostly I’m guessing due to the more streamlined login, but for me the strong security guarantees are the compelling part.
People are onboarded by what looks like a more convenient login but I (and they probably too) have no idea about the edge cases. What happens if they lose a device? No way to figure this out really without going through it.
Most people with serious security requirements seem to be using Yubikeys anyway. For most normal people passwords written in a paper book are the best solution, and critical digital logins backed by some national identity or notary scheme so they can retrieve their accounts in the case of loss.
(I just got locked out of an old Google account because I don’t have access to the 2FA phone number anymore. Let’s just say I’m glad there wasn’t really anything of value in there.)
That’s what the backup codes that Google prompts you to save in a secure place after setting up 2FA are for.
You don’t say! Thank you for the explanation.
It of course perfectly illustrates the failure modes of these systems and if I a CS graduate am bit by this, then for sure most normal people do not stand a chance.
Exactly. There is a reason why accessibility is part of the common understanding of security.
It’s too bad every large website implementing passkeys has figured out that you still have to support email-based self-service recovery to avoid a flood of support requests.
All of these protections are pointless if every passkey can be bypassed through email. Attackers have no problem walking users through recovery flows.
Suppose I have a site that I usually log into from my phone, but today I want to connect to it from a browser on my mother’s computer. Is that simply prohibited in the name of better security? Or is there a way I can accomplish it if there is no password, only a passkey.
That’s up to the site, you’re fundamentally in a position of doing a new device log in, so you’d presumably need a full 2fa dance, which they could permit via your signed in phone, or whatever other mechanism they choose. However they would still have to take steps to mitigate the risk of that being used as a phishing/social engineering attack (you basically start to run the risk of creating the kind “account recovery” style mechanism scammers use to compromise accounts).
It seems to me that there is an irreconcilable issue here: either a site allows some kind of account recovery (and “log in from my mother’s computer” is an example of “account recovery”), in which case there is a risk of it being used in a social engineering attack, or it does NOT allow any kind of account recovery, in which case accounts can’t be recovered.
What I would like to see is some enhanced forms of validation needed when doing “account recovery”. But that ends up meaning that I CANNOT “get rid of passwords and just use passkeys instead” (which is the promise) unless I also give up completely on having ways to recover an account.
Have to? Why is it the site’s responsibility to prevent technically illiterate users from giving their password to someone else?
Because it’s not just technically illiterate users? There are many attack vectors that are not trivial to identify and human brains are notoriously bad at reading/seeing what they expect to see. Yes of course there are people who are more gullible than others, but the security of your account should depend on more than just never ever making a mistake.
A better question there would be “why should be able to use your service securely be dependent on expert level understanding of attacks?”
If as you say the problem is just not being technically literate I’m curious if you avoid/ignore 2FA mechanisms, because those exist simply to try and provide some of the guarantees you get from passkeys. If it’s just a matter of technical literacy then it’s just an annoyance during log in.
Thanks for an explanation, it is helpful!
i’m struggling to see how transferable passkeys aren’t just isomorphic to passwords in a password manager.
i suppose with certain passkey schemes you could perform a challenge-response authentication over an unencrypted channel… but i’m not sure that’s desirable tbh.
Any “move from service A to service B” is going to involve a cryptographic exchange, e.g. service A would re-wrap the key material to a key provided by service B (this is the “don’t leak the unwrapped key material”). But I agree, my concern is that supporting that opens a vector for scammers to convince people to transfer their keys to an attacker controller application which obviously is not going to care too much about actually keeping the data secret, which is a significant reduction in the security against social engineering (in that it goes from being actually impossible to being at all possible).
It is still important to understand though that even with that obnoxious problem passkeys are still fundamentally different from passwords, the reason they’re set up the way they are is to prevent any kind of man in the middle or social engineering attack. They’re not just an HSM backed password manager (that’s a bare minimum requirement of any password manager), they’re cryptographic keys that allow authentication that also proves to a server that the client performing the authentication is the client it is talking to. That’s not something any password or password+2fa/totp authentication can do, and that’s a weakness that is constantly being exploited.
This is a legitimate concern, but I think the UI challenges are solvable. First, unlike copying and pasting a password, this is not going to be a common flow. People who are happy with a single vendor’s ecosystem will never use it and the flows between the big vendors are likely to be polished.
Even without multi-vendor support, the Apple flow is pretty good. You need to log into the new device with your Apple ID. This requires a password for bootstrapping but then requires a second factor p, which is typically another Apple device that tells you the device that you’re trying to log into and where it is. The latter is the biggest UI flaw because Apple’s geolocation is terrible and seems to be no more accurate than getting the correct country, but that should be fixable. You could also add a distance-bounding protocol over Bluetooth in here for the location-looks-wrong case.
This is a flow that users will do once when setting up a new device and can be flagged with ‘this is a new device setup thing, make sure that the target is your device, never let anyone else do this’ warning.
The bigger worry for me is coercion. If you have a person and their phone, forcing them to sync their passkeys to your device is quite easy. WebAuthn doesn’t seem to have a good revocation story for ‘this copy of my keys is a stolen one, please undo everything it’s done and lock it out’. Windows Hello has device-bound credentials, which carry the device identity with them so you can tell the difference between logins by the same user from different devices, but I believe that’s a non-standard extension.
isn’t this already handled by oauth? or am i misunderstanding what you mean here by “service”?
i’m really just struggling to see what “the point” is. it basically seem like making it slightly easier to follow good security practices at the cost of making it significantly harder to actually use services.
Yes, but I think entirely due to bad explanation on my part. It’s a skill :D
What I’m talking about here is the passkey provider, e.g apple’s keychain, googles , 1password, etc
Something people constantly complained about with passkeys was you create your passkeys with Service A (say keychain) and you can’t simply transfer that passkey to a different tool like 1password, what you had to do was create a new passkey with the new provider you wanted to use instead. For a couple of sites that’s not an issue, but if you were say just planning to transition to a new ecosystem and wanted to move all of your credentials over that’s a lot of work.
The thing is that the reason passkeys did not permit the extraction/transfer of the material is because scammers are exceptionally good at convincing their victims to do things, like telling the person on the phone what their 2fa token is, or installing software and giving that software excitingly elevated permissions.
Introducing the ability to export passkeys to another provider opens the previously non existence option of an attacker convincing their victim to install a piece of software and then export their passkeys to it, and in doing so provide all the material to the scammer.
No TLS does not resolve this kind of man in the middle, as this isn’t just about MiTM of a network connection, it MiTM of the authentication path. The user is convinced, by whatever mechanism, to open a scam page. The url is not the real website, but it is objectively true that users are bad at recognizing this. The scammers are forwarding the content of the real web page to the user of the fake url, the user enters there password in the fake page (sites are good enough at breaking password autofill that even users who are using password managers routinely fall for this), the scam page sends that back to attacker who uses that to log in. If the site has some form of 2FA, the scammer simply forwards that request to the victim, the user is expecting this, so still does not question the need and enters the code, which is forwarded to the scammer, who then has successfully compromised the account.
Passkeys tie the authentication to the actual identity of page being loaded on the authenticating machine, so this kind of MiTM simply does not work.
All the social engineering attacks targeting authentication revolve around getting the victim to provide their passwords and necessary 2FA tokens to the attacker, which only works because the passwords and 2fa tokens are not tied to the device or connection being authenticated.
These are the ways that scammers compromise accounts in the real world, and passkeys prevent them in a way that is fundamentally not possible via any other mechanism (in the sense of “this authentication model is the only way to prevent these kinds of attacks”, “passkeys” just happen to be the implementation+branding we have for the model).
Ah, this addresses my wondering why commenters are saying password managers are so insufficient. (I hadn’t personally encountered broken password autofill not being a reasonably reliable signal that one’s not talking to the right website.)
it’s super annoying, especially now so many sites have started breaking sign in pages into multiple steps, something that serves literally no purpose other than breaking automatic sign in and encouraging users to expect random page loads and page changes during sign in. Le sigh.
Still at least there’s fairly standard mechanisms for marking a password field so the autofill is actually possible, as opposed to the relentlessly stupid and brain dead password rules and restrictions that sites keep inventing, despite decades of research showing the outcome is worse passwords that are reused, and systematically selected. Mercifully NIST has finally updated their docs to say “stupid ass rules that prevent random passwords are stupid and you suck” :D
In my limited experience of websites with multi-stage sign-in pages, the Google Chrome password manager copes with them, even if it needs to have the user choose from the saved username/password pairs (if there are >1) at the username step and then again at the password step, which still keeps the passwords that the user is prompted to use limited to ones associated with the domain. I don’t doubt that there are worse sign-in pages with which I’m not familiar, though!
Oh yeah, I think they all manage it now, it’s just more work for the implementation (the site navigation and UI changes makes it much harder to link account and password fields). For autofill you have to approve the autofill multiple times, etc.
It literally just makes site login slower, password autofill more annoying to implement, and trains users to expect jank and weird page loads and refreshes scattered throughout a legitimate login.
It’s just a very weird and obnoxious decision by these sites.
It doesn’t if the username and password pages are served on separate random subdomains, which sso.duosecurity.com helpfully does for its customers.
Yes, but nobody uses client certificates, which would be the relevant equivalent part of TLS here.
As for social engineering: yeah, social engineers just have to go through the account recovery/new device enrollment flow.
But if you want to be able to share keys around and back them up without some third-party service (which may be niche requirements, but they are critical to me) then you need some form of serialized secret which is effectively a password. At that point you may make it user-readable and easy to type on a keyboard. Sure, I almost never do type it. But when I do need to it is invaluable.
Passkeys are not passwords that are being kept secret from you, they’re a cryptographic challenge/response process.
They’re not human typable because passkey authentication means signing a server provided nonce as proof of identity, and the server is able to verify that the machine doing the authentication is yours because an attacker is not able to get your machine to sign the data that the server has given them. I tried to summarize in my response here https://lobste.rs/s/2mgwsz/passwords_have_problems_passkeys_have#c_mp7x0o
I should have been more clear that I am talking about the storage format not the authentication protocol.
You can also do challenge response with passwords (ex: SRP, OPAQUE). My opinion is that upgrading the authentication protocol without moving away from passwords as a secret format would have been a better approach. Password managers can become more aggressive about warning users when displaying and exporting passwords (to prevent phishing and similar) but we don’t throw away the concept of passwords that is so simple to understand that my grandparents can back up their password manager into a notebook.
Basically I feel that we gave up so much for an improvement that could have been added to the simple system. With a PAKE and some UX you get basically all of the benefits of passkeys but users can still understand how it works to a degree that lets them do useful things like authenticate on a friend’s machine.
This seems like the core disagreement here: some view this as an important UX feature, and some consider this to be a security bug in need of prevention.
Both sides have their point! And it indeed seems that it’s not actually possible to automatically separate useful “your friend’s machine” from malicious fishing.
I wish someone has stated the tradeoff more explicitly here. It is clear what attacks are prevented. It is not completely clear which valid use-cases are prevented as a collateral damage.
The problem is less a disagreement but rather one side (the passkey side) trying to push their opinion onto the other side. I’m okay to have the option to use passkeys in addition to passwords. The passkey folks don’t seem to like it, they need to replace passwords.
I mean, in this very thread there are some comments that say that passkeys have no advantage over passwords in a password manager, and the article itself kinda-sorta claims the same.
Things are very rarely one-sided, even if it looks like that from a particular side!
Can you link to one of those comments?
DMed!
How does PAKE address the benefits of pass keys? I don’t think it does at all. You can MITM PAKE just as easily as any password, it’s just that the password doesn’t get leaked - the signed response does.
PAKE is great because the underlying secret never gets conveyed but it doesn’t address phishing. PAKE’s main benefit is that you don’t end up with “mypassword1234” scattered in your nginx logs or whatever.
Also, PAKE is not straightforward to implement on the server. I know that passkeys aren’t either, but I do just want to point out that it’s not simple. User signup becomes much trickier.
Ah ok, I think I understand what you were saying now, but to confirm, in essence passkeys perform authentication by
but you’re saying you’d prefer
which is fundamentally equivalent assuming the secret password has as much entropy as the cryptographic key.
Is that correct? At least as a general approximation?
Yes. I think you get my point.
Ok, in that case the issue just becomes the non-escaping nature of (the original) passkey model.
The reason for that mechanism is specifically to prevent scammers from being able to get people to provide the actual secret, because once you provide the secret to the attacker they can authenticate with the material the server provides their own connection.
The reason folk want a password, is because they want to be able access the secret, and in other words defeat the above.
I guess a good way to think about this is, how would it impact your opinion if the passwords from your password manager were just base64 encoded ECC keys?
I believe that you can only protect people from themselves so much. This is why I would rather make it a UX issue than add technical limitations and proprietary sync protocols. We can make the UX very aggressive about letting people know that they are doing something risky. (Or for corporate secrets actually block exporting).
If they were of a reasonable length then I would be fine with it. Honestly if a portable and reasonably human-manageable export format for passkeys was standardized would be pretty fine with the whole system. Maybe it will land some day, but I feel that this is a critical feature that should have been mandated from day one.
If there is a well understood, and effective, attack that works consistently, and you can simply make that attack not function why would you not?
right, so your issue is not the authentication scheme, which is superior to password based auth, its that you can’t simply extract the key material and use it directly yourself. Despite not being able to do any of the authentication yourself - it’s not an operation any person is doing in their head - and it means you’ve taken on the burden of verifying the host identity of the forms you’re reading and filling out (which is not necessarily the same as “what is the url the location bar shows”)
Because it’s at the expense of being able to do something useful (login on a device you do not own or where you haven’t synced your passkeys yet).
It’s at the expense of friction, not ability. You can still log in. If my phone is my passkey I can have my phone authorize me via, for example, a QR code.
Because there are downsides that are not worth it.
We can lock everyone in their own house to prevent murder, STDs and many other things. But it is not worth the tradeoff. We can force people to eat a regimented diet to prevent a huge number of illnesses. But that isn’t a world I want to live in. We should help people to the right thing, but at some point we need to give people freedom.
Really, it’s fine for me if the passwords from my password manager are just base64 encoded ECC keys. I just want to be able to use my choice of password manager, and sync its encrypted database file however I please. I’m not hugely concerned if there is a standard way for the browser and the password manager to communicate that’s not copy-and-paste, either. What I am concerned about is having my choice of password manager constrained to e.g. Apple or Google.
It isn‘t with Passkeys. 1Password implements them, and any other password manager can too. Apple, and I guess Google and Microsoft also, provide APIs to hook into both password autofill and passkeys for 3rd party managers to use.
This release broke Datasette due to one of my dependencies not working with Python 3.13 yet - I pushed an update just now that fixes that, details of the change in this issue.
Huh, why do the Python developers make backwards incompatible changes?
Usually they don’t without several years of warning.
It looks like in this case they were fixing a bug: https://github.com/python/cpython/issues/109409
Unfortunately my Pint dependency was relying on that bug.
FYI: Python does not use SemVer and never really has. There’s always been a rolling deprecation cycle in Python releases, and it’s always been important to pay attention to deprecation warnings and release notes.
As to why backwards-incompatible changes get made at all, it’s because the alternative is to be something like Java, where there are decades of accumulated “no, don’t use that API for this, or that one, or that one, or that one” footguns because an absolute obedience to never making a breaking change prevents the removal of known-bad APIs.
Doesn’t Python have a bunch of known crappy modules in its standard library?
Python’s been working on removing a bunch of outdated modules/APIs in its standard library and replacing them with or directing people to better alternatives.
Of course, doing so breaks backward compatibility, which people then complain about.
Yup, we’re having fun in the trenches.
I think there’s a difference between “this outright doesn’t work” (which I think is mainly a thing in something like
datetime.replacejust doing the ‘wrong’ thing when you pass in a timezone for example), and “this is not a nice API”, which I think is how I would qualify some standard library APIs.And even then, honestly? The standard library is pretty fine in my experience. The main complaint I’d have is
urllibstuff just not really working out of the box on Mac without shenanigans to get certs working first.Funny you mention datetime… here are a few pitfalls:
https://dev.arie.bovenberg.net/blog/python-datetime-pitfalls/
Referenced via
whenever:“Do you cross your fingers every time you work with Python’s datetime—hoping that you didn’t mix naive and aware?”
https://github.com/ariebovenberg/whenever
In my experience, everyone gets dates and time wrong, including me. The implicit assumptions list in date and time math is 400 miles long when printed in size 1 font..
I’ve never looked at whenever, but to believe it fully encompasses the date and time problem is probably ridiculous. When countries can’t even agree on what the right date or time is at a given place, and cultures/groups of people sometimes have their own date and time interpretation separate from their countries, there is zero hope anyone can get it right in a library.
Fair enough. I have the luck of both living in places without DST and also knowing to never use naive datetimes, so most of these are non-issues for me. But it would be nice for all of this stuff to just be…. better.
datetimes are total footguns in general if you just open up the standard library and use it.
I’m happy at least tzinfo is easier to get a hold of than it used to.
Removing quite a few of those ancient modules was part of this release!