People are asking what formal methods Amazon is using. My understanding (as a non-Amazon person) is that the answer is : “a bit of everything”. They have been hiring people with expertise is automated theorem provers (the main Z3 author moved to Amazon), but also verified-programming systems like Dafny (the main author moved to Amazon), proof assistants, several of them (Hol Light, Coq, Lean), modelling tools like TLA+, etc. A bit of everything. From the outside it feels like they have many different (sub)teams interested in formalization problems or many different sub-problems that justify them, and people have a lot of freedom to pick the approach they think is best as long as it delivers result. They are also funding collaboration with academia to try out other approaches, new tools, etc.
There are more details for example in this PDF document which comes from one of those team, and mentions again many different tools for formal verification.
Assuming you’re referring to Leonardo de Moura, he was involved in the early days from 2012-2014 (self-describes as the main architect) but since then he moved onto developing Lean and Nikolaj Bjørner has been the principal developer along with Lev Nachmanson and Christoph Wintersteiger.
A historical note: Z3 is older than that. The classic paper on it places its first release in 2007, and it’s been leading the SMT competition for a very long time. I expect that development must have started somewhere between 2003 and 2005, but don’t quote me on that. Work on Lean started in 2013 so that must be roughly when Leo de Moura started working on it.
Yes, that’s pretty accurate. Bit of everything, whatever we need to solve the business problem.
In addition, we’ve got some internal tools that aren’t available externally (yet), and an increasing investment in combining AR techniques with neural/AI/LLM/whatever you wanna call it techniques. Missing from your list on the modelling tools side is P (https://p-org.github.io/P/whatisP/), which is also primarily developed at Amazon right now, and is widely used internally for distributed systems stuff.
From the outside it feels like they have many different (sub)teams interested in formalization problems or many different sub-problems that justify them, and people have a lot of freedom to pick the approach they think is best as long as it delivers result.
This has, historically, been AWS’s approach to this kind of “researchy” work. We’ve really tried to optimize for technology transfer, making sure the benefits of the work end up in production and benefiting customers, over some other concerns. I think that’s been very successful on the whole, although isn’t without its trade-offs compared to other models.
Agreed! I am really interested in this stuff. The double whammy promise of faster and Correct with a capital C is attractive. Are they using anything besides TLA+?
It is weird that they don’t mention specific technologies, but the S3 work they talked about used TLA+, the IAM work used Z3 (or at least a modified version called Zelkova), and the cryptography work used an in-house interactive prover called HOL Light. I know they also use P and have heard rumors of something similar to P but for model-checking rust code. Don’t know what either of those are used for.
I’m told that zelkova is mostly backed by cvc5 (Amazon employs/funds a bunch of cvc5 researchers, among other things for string/regex reasoning). Dafney, however, is still backed by Z3 by default.
HOL light is not “in house” but was developed for a long time before AWS hired John Harrison. Previously I think he was at Intel. I think it’s been used to verify cryptographic primitives at AWS, but previously it was also used for hardware verification.
People are asking what formal methods Amazon is using. My understanding (as a non-Amazon person) is that the answer is : “a bit of everything”. They have been hiring people with expertise is automated theorem provers (the main Z3 author moved to Amazon), but also verified-programming systems like Dafny (the main author moved to Amazon), proof assistants, several of them (Hol Light, Coq, Lean), modelling tools like TLA+, etc. A bit of everything. From the outside it feels like they have many different (sub)teams interested in formalization problems or many different sub-problems that justify them, and people have a lot of freedom to pick the approach they think is best as long as it delivers result. They are also funding collaboration with academia to try out other approaches, new tools, etc.
There are more details for example in this PDF document which comes from one of those team, and mentions again many different tools for formal verification.
By the way, the main HOL Light author (John Harrison) also moved to Amazon.
Assuming you’re referring to Leonardo de Moura, he was involved in the early days from 2012-2014 (self-describes as the main architect) but since then he moved onto developing Lean and Nikolaj Bjørner has been the principal developer along with Lev Nachmanson and Christoph Wintersteiger.
A historical note: Z3 is older than that. The classic paper on it places its first release in 2007, and it’s been leading the SMT competition for a very long time. I expect that development must have started somewhere between 2003 and 2005, but don’t quote me on that. Work on Lean started in 2013 so that must be roughly when Leo de Moura started working on it.
Yes, that’s pretty accurate. Bit of everything, whatever we need to solve the business problem.
In addition, we’ve got some internal tools that aren’t available externally (yet), and an increasing investment in combining AR techniques with neural/AI/LLM/whatever you wanna call it techniques. Missing from your list on the modelling tools side is P (https://p-org.github.io/P/whatisP/), which is also primarily developed at Amazon right now, and is widely used internally for distributed systems stuff.
This has, historically, been AWS’s approach to this kind of “researchy” work. We’ve really tried to optimize for technology transfer, making sure the benefits of the work end up in production and benefiting customers, over some other concerns. I think that’s been very successful on the whole, although isn’t without its trade-offs compared to other models.
Interesting but very light on details beyond the headline.
Yeah, I agree… it seems like they might be using HOL Light, from an article linked in the post?
Agreed! I am really interested in this stuff. The double whammy promise of faster and Correct with a capital C is attractive. Are they using anything besides TLA+?
It is weird that they don’t mention specific technologies, but the S3 work they talked about used TLA+, the IAM work used Z3 (or at least a modified version called Zelkova), and the cryptography work used an in-house interactive prover called HOL Light. I know they also use P and have heard rumors of something similar to P but for model-checking rust code. Don’t know what either of those are used for.
Some corrections about this:
I flagged this as spam, it is just marketing fluff.