DSHR's Blog

Software Liability: US vs. EU

2025-02-27T08:00:00.000-08:00

I have written before about the double-edged sword of software vendors' ability to disclaim liability for the performance of their products. Six years ago I wrote The Internet of Torts about software embedded in the physical objects of the Internet of Things. Four years ago I wrote about Liability In The Software Supply Chain.

Source

Last October, Tom Uren wrote The EU Throws a Hand Grenade on Software Liability:

The EU and U.S. are taking very different approaches to the introduction of liability for software products. While the U.S. kicks the can down the road, the EU is rolling a hand grenade down it to see what happens.

It is past time to catch up on this issue, so follow me below the fold.

USA

Source

In March 2020 the Cyberspace Solarium Commission, a "bipartisan team of lawmakers and outside experts", launched their report. Among its 82 recommendations were several related to liability:

3.3.2: Clarify Liability for Federally Directed Mitigation, Response and Recovery Efforts
4.2 Congress should pass a law establishing that final goods assemblers of software, hardware, and firmware are liable for damages from incidents that exploit known and unpatched vulnerabilities
4.3 Congress should establish a Bureau of Cyber Statistics charged with collecting and providing statistical data on cybersecurity and the cyber ecosystem to inform policymaking and government programs

Source

In March 2023 the Biden administration announced their National Cybersecurity Strategy. As regards liability, it stated:

Markets impose inadequate costs on — and often reward — those entities that introduce vulnerable products or services into our digital ecosystem. Too many vendors ignore best practices for secure development, ship products with insecure default configurations or known vulnerabilities, and integrate third-party software of unvetted or unknown provenance. Software makers are able to leverage their market position to fully disclaim liability by contract, further reducing their incentive to follow secure-by-design principles or perform pre-release testing. Poor software security greatly increases systemic risk across the digital ecosystem and leave[s] American citizens bearing the ultimate cost.

What did the administration propose to do about this?:

The Administration will work with Congress and the private sector to develop legislation establishing liability for software products and services. Any such legislation should prevent manufacturers and software publishers with market power from fully disclaiming liability by contract, and establish higher standards of care for software in specific high-risk scenarios. To begin to shape standards of care for secure software development, the Administration will drive the development of an adaptable safe harbor framework to shield from liability companies that securely develop and maintain their software products and services. This safe harbor will draw from current best practices for secure software development, such as the NIST Secure Software Development Framework. It also must evolve over time, incorporating new tools for secure software development, software transparency, and vulnerability discovery.

In October 2024 Eric Geller looked at what happened to the Cyberspace Solarium Commission's recommendations in The struggle for software liability: Inside a ‘very, very, very hard problem’:

Six years after Congress tasked a group of cybersecurity experts with reimagining America’s approach to digital security, virtually all of that group’s proposals have been implemented. But there’s one glaring exception that has especially bedeviled policymakers and advocates: a proposal to make software companies legally liable for major failures caused by flawed code.

It wasn't like the Commission invented the liability problem:

Since the 1980s, legal scholars have discussed how liability should apply to flawed software. The fact that there still isn’t a consensus about the right approach underscores how complicated the issue is.

One of the biggest hurdles is establishing a “standard of care,” a minimum security threshold that companies could meet to avoid lawsuits. There’s disagreement about “how to define a reasonably secure software product,” Dempsey said, and technology evolves so quickly that it might not be wise to codify one specific standard.

Various solutions have been proposed, including letting juries decide if software is safe enough — like they do with other products — and letting companies qualify for “safe harbor” from lawsuits through existing programs like a government attestation process.

The Solarium Commission proposed safe harbor for companies that patch known vulnerabilities. But that would only address part of the problem.

Even a very weak "duty of care" would be a big improvement. It would, for example, outlaw hard-wired passwords, require 2-factor authentication with FIDO or passkeys not SMS, require mailers to display the actual target of links, and so on.

The industry pushback has been ferocious:

One of the industry’s chief arguments is that liability would distract companies from improving security and overburden them with compliance costs. “The more companies are spending their time on thinking about liability, the less they might be spending their time on higher-value activities,” said Henry Young, senior director of policy at the software trade group BSA.

The problem is that the "higher-value activities" typically result in adding vulnerabilites to their products. I doubt that management at victim companies like Equifax or SolarWinds' customers would think that adding flashy new features was "higher-value" than fixing vulnerabilities.

You can't say the industry's flacks aren't creative:

Liability opponents say insecure software isn’t the biggest cybersecurity problem, pointing to widespread and devastating phishing attacks.

Riiiiight! Phishing is a force of nature, not a tactic enabled by the industry's insecure products.

Some of the industry's arguments are laughable:

They argue that even if policymakers want to focus on software security, there are better ways to prod vendors forward, such as encouraging corporate-board oversight.

Board members are insulated from liability by D&O insurance, so "encouraging" them will have precisely zero effect.

But this is my favorite:

And they warn that focusing on liability will distract the government from pursuing better policies with its limited resources.

Exactly what are the "better policies" they desire? To be left alone to ship more buggy products.

The industry whines that they wouldbe treated differently from others, conveniently ignoring that the others can't disclaim liability:

Critics also contend that it’s unfair to punish companies for digital flaws that are deliberately exploited by malicious actors, a scenario that’s rare in most industries with liability, such as food and automobiles.

In 2023 nearly 41,000 people were killed by automobiles in the US. Many, probably most of those deaths were caused by "malicious actors" exploiting flaws such as cars that know what the speed limit is but don't enforce it, or could but don't detect that the driver is drunk or sleepy. Liability hasn't caused the auto industry to get real about safety. Instead we have Fake Self-Driving killing people. And:

Up to 48 million people get sick from a foodborne illness every year, and up to 3,000 are estimated to die from them.

Finally, the industry claims that everything in just fine:

Industry leaders say liability is unnecessary because there’s already a working alternative: the marketplace, where businesses are accountable to their customers and invest in security to avoid financial and reputational punishment. As for contracts disclaiming liability, the industry says customers can negotiate security expectations with their vendors.

“We're open to conversations about any way to improve software security,” Young said. “Our customers care about it, and we want to deliver for them.”

Have you tried to "negotiate security expectations" with Microsoft, or have a conversation with Oracle about a "way to improve software security"? How did it go? I guess it didn't go well:

“Just telling organizations that not fixing security bugs will impact their business is not enough of an incentive,” a group of tech experts warned the Cybersecurity and Infrastructure Security Agency in a report approved this month.

Experts also rejected the idea that most customers could negotiate liability into their contracts. Few companies have leverage in negotiations with software giants, and few customers know enough about software security to make any demands of their vendors.

Despite its laughable nature, the industry's pushback ensured that nothing happened until it was too late:

Senior administration officials haven’t lived up to their lofty rhetoric about shifting the burden of cybersecurity from customers to suppliers, Herr said. “There is an attitude in this White House of a willingness to defer to industry in operational questions in a lot of cases.”

In the end it may come down to judges to determine whether the disclaimers of liability are effective, and there may be a siight ray of hope. Last December Sean Lyngaas reported that Judge rules Israeli firm NSO Group liable for damages in WhatsApp hacking case:

Messaging service WhatsApp claimed a major legal victory over Israeli spyware firm NSO Group on Friday after a federal judge ruled that NSO was liable under federal and California law for a 2019 hacking spree that breached over 1,000 WhatsApp users.

It’s a rare legal win for activists who have sought to rein in companies that make powerful spyware, or software capable of surveilling calls and texts, that has reportedly been used on journalists, human rights advocates and political dissidents worldwide.

So in the US the disclaimers of liability in the end user license agreement are, and will presumably continue to be, valid unless your product is intended to commit crimes such as violations of the Computer Fraud and Abuse Act. The bar for a victim to prove liability is impossibly high.

Color me skeptical, but Jim Dempsey writes in The MAGA Case for Software Liability:

Under the current administration, the instinctive inclination of post-Reagan Republicans to rely only on market forces to hold businesses responsible for the consequences of their actions would seem to preclude the use of government policy to improve the security of software vital to business and government operations. Indeed, the Trump team has promised wholesale repudiation of regulations adopted in the past four years, so new limits on industry would seem especially unlikely.

There are, however, good reasons why the new administration should not default to repealing the cybersecurity actions of the past four years and passively accepting severe cyber vulnerabilities in critical infrastructure. In fact, as I explained in a series last year on initiatives aimed at infrastructure and data, much of the Biden administration’s cybersecurity agenda was built on projects launched by President Trump in his first term. The Trump administration would do well to remember the underlying principles that spurred it to initiate these actions the first time around.

EU

Source

The EU seems to be taking an opposite approach. Tom Uren wrote:

Earlier this month, the EU Council issued a directive updating the EU’s product liability law to treat software in the same way as any other product. Under this law, consumers can claim compensation for damages caused by defective products without having to prove the vendor was negligent or irresponsible. In addition to personal injury or property damages, for software products, damages may be awarded for the loss or destruction of data.

In the EU companies are presumed to be liable for a defective product unless they can qualify for a "safe harbor":

Rather than define a minimum software development standard, the directive sets what we regard as the highest possible bar. Software makers can avoid liability if they prove a defect was not discoverable given the “objective state of scientific and technical knowledge” at the time the product was put on the market.

The directive is based on 2 main principles:

the manufacturer has to compensate the damage caused by a defective product of theirs

the victim has to prove the product’s defectiveness, the damage that was caused and establish that this defectiveness was the cause of the damage

And the products are defined thus:

Digital economy: The new law extends the definition of “product” to digital manufacturing files and software. Also online platforms can be held liable for a defective product sold on their platform just like any other economic operators if they act like one.

Circular economy: When a product is repaired and upgraded outside the original manufacturer’s control, the company or person that modified the product should be held liable.

Unlike the current and proposed US approaches, the EU's approach imposes victim-driven consequences for:

Failure to use current software tools and development practices to prevent defects. Note that the directive gives victims the right to obtain evidence of this kind.
Failure to acknowledge and respond to defect reports from customers and third parties, because they were clearly discoverable via the "objective state of scientific and technical knowledge". But note the caveat "at the time the product was put on the market".
Failure to issue timely fixes for defects.

Uren concludes:

Major software vendors used by the world’s most important enterprises and governments are publishing comically vulnerable code without fear of any blowback whatsoever. So yes, the status quo needs change. Whether it needs a hand grenade lobbed at it is an open question. We’ll have our answer soon.

Open Source

Neither the US nor the EU approaches seem to take account of the fact that many of the "products" they propose to regulate are based upon open source code. An under-appreciated feature of the rise of IT has been the extraordinary productivity unleashed by the 1988 Berkeley Software Distribution license and Richard Stallman's 1989 GNU General Public License. Version 3 of the Gnu Public License Section 15 states:

THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

And Section 16 states:

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

That these licenses disclaimed warranties and liabilities was the key to the open source revolution because it enabled individuals, rather than developers protected by their employer's lawyers, to contribute. Without the disclaimers individual developers would face unacceptable legal risk. On the other hand, open source is also a channel for malware. Shaurya Malwa reports on an example in Hackers Are Using Fake GitHub Code to Steal Your Bitcoin: Kaspersky:

The report warned users of a “GitVenom” campaign that’s been active for at least two years but is steadily on the rise, involving planting malicious code in fake projects on the popular code repository platform.

The attack starts with seemingly legitimate GitHub projects — like making Telegram bots for managing bitcoin wallets or tools for computer games.

Each comes with a polished README file, often AI-generated, to build trust. But the code itself is a Trojan horse: For Python-based projects, attackers hide nefarious script after a bizarre string of 2,000 tabs, which decrypts and executes a malicious payload.

For JavaScript, a rogue function is embedded in the main file, triggering the launch attack. Once activated, the malware pulls additional tools from a separate hacker-controlled GitHub repository.

Balancing the need to stop vendors "publishing comically vulnerable code" with the need to nurture the open source ecosystem is a difficult problem.

On Not Being Immutable

2025-02-06T08:00:00.000-08:00

Economist 2/1/25

Regulation of cryptocurrencies was an issue in last November's US election. Molly White documented the immense sums the industry devoted to electing a crypto-friendly Congress, and converting Trump's skepticism into enthusiasm. They had two goals, pumping the price and avoiding any regulation that would hamper them ripping off the suckers.

Back in November of 2022 I added an entry to this blog's list of Impossibilities for The Compliance-Innovation Trade-off from the team at ChainArgos. It started:

tl;dr: DeFi cannot be permissionless, allow arbitrary innovation and comply with any meaningful regulations. You can only choose two of those properties. If you accept a limited form of innovation you can have two-and-a-half of them.

Fundamental results in logic and computer science impose a trade-off on any permissionless system’s ability to both permit innovation and achieve compliance with non-trivial regulations. This result depends only on long-settled concepts and the assumption a financial system must provide a logically consistent view of payments and balances to users.

This is a semi-technical treatment, with more formal work proceeding elsewhere.

Two years later, the "more formal work" has finally been published in a peer-reviewed Nature Publishing journal, Scientific Reports, which claims to be the 5^th most cited journal in the world. Jonathan Reiter tells me that, although the publishing process took two years, it did make the result better.

Below the fold I discuss Tradeoffs in automated financial regulation of decentralized finance due to limits on mutable turing machines by Ben Charoenwong, Robert M. Kirby & Jonathan Reiter.

This team were pioneers in applying fundamental computer science theorems to blockchain-based systems, starting in April 2022 with The Consequences of Scalable Blockchains in which they showed that implementing an Ethereum-like system whose performance in all cases is guaranteed to be faster than any single node in the network is equivalent to solving the great unsolved problem in the theory of computation, nicknamed P vs. NP. And thus that if it were implemented, the same technique could break all current cryptography, including that underlying Ethereum.

But, I believe, they were not the first. That appears to have been Tjaden Hess, River Keefer, and Emin Gün Sirer in Ethereum's DAO Wars Soft Fork is a Potential DoS Vector (28^th June 2016), which applied the "halting problem" to "smart contracts" when analyzing possible defenses against DOS attacks on a "soft fork" of Ethereum proposed in response to "The DAO".

Charoenwong et al's abstract states:

We examine which decentralized finance architectures enable meaningful regulation by combining financial and computational theory. We show via deduction that a decentralized and permissionless Turing-complete system cannot provably comply with regulations concerning anti-money laundering, know-your-client obligations, some securities restrictions and forms of exchange control. Any system that claims to follow regulations must choose either a form of permission or a less-than-Turing-complete update facility. Compliant decentralized systems can be constructed only by compromising on the richness of permissible changes. Regulatory authorities must accept new tradeoffs that limit their enforcement powers if they want to approve permissionless platforms formally. Our analysis demonstrates that the fundamental constraints of computation theory have direct implications for financial regulation. By mapping regulatory requirements onto computational models, we characterize which types of automated compliance are achievable and which are provably impossible. This framework allows us to move beyond traditional debates about regulatory effectiveness to establish concrete boundaries for automated enforcement.

They summarize the fundamental problem for the automation of DeFi regulation:

DeFi features some computationally challenging properties: (1) Turing-complete programming, (2) permissionless access to both transact and publish code and (3) selectively immutable code. The permissionless mutability of the system combined with the Turing completeness motivates our inquiry. A system running Turing-complete code where updates can be published permissionlessly cannot make any guarantees about its future behavior, a conclusion from early work on Universal Turing Machines (UTM).

Despite this, they show that if it is possible to enforce less-than-Turing-complete programming:

it is possible to construct both (1) classes of algorithms that can make credible promises and (2) restricted update mechanisms that enable credible promises. In other words, DeFi platforms can provide compliant services like traditional centralized providers through fully automatic mechanisms.

What exaxctly do they mean by "compliant"?

Consider an economy modeled as a Turing Machine, where the machine’s state corresponds to the state of the real economy. We formalize compliance as a property of system state transitions that can be verified mechanically, following Theorem 5.8.5 of Savage. Specifically, a compliant system is one where no sequence of permitted operations can result in a state that violates predefined regulatory constraints set by an external regulator.

They translate this into economic terms:

For example, if a regulation prohibits transactions with certain addresses, compliance means no sequence of permitted operations can transfer value to those addresses, either directly or indirectly. Similarly a regulator may impose requirements on intermediaries transacting in certain assets or products akin to depository receipts for those assets. Compliance would then require ensuring one does not unknowingly transact in “products akin to depository receipts” for a given list of assets.

Because they model regulated systems in terms of states and the transitions between them, they can apply results from compputer science:

This formulation maps to well-known results in computability, such as the Halting Problem and the more general impossibility known as Rice’s Theorem: No algorithm exists to determine from the description of a [Turing Machine] whether or not the language it accepts falls into any proper subset of the recursively enumerable languages. In other words, we cannot categorize arbitrary programs into specific subsets automatically and reliably. In financial regulation, the canonical “proper subset” is a ban on interacting with a given address: interactions involving a banned address are forbidden, and the acceptable subset of states includes no such transfers.

They proceed to use cryptocurrency "mixers" as an example. They explain that Rice's Theorem means that because any general description of "mixers" defines a "proper subset" of all programs, and thus there is no automatic or reliable method by which a "smart contract" can be assessed against the general description.

Of course, as Justice Potter Stewart famously said "I know it when I see it". Human regulators will have no difficulty in recognizing a "smart contract" as a mixer, if only because in order to function it needs liquidity. To attract it the "smart contract" needs to advertise its service. So can the regulators ban interactions with the specific addresses of the mixers they recognize? Charoenwong et al make two arguments

First, this severely limits the regulator’s power from regulating a mutable set of protocols to only specific ones. In other words, what is often called “principles-based” regulations (as opposed to rules-based regulations) are impossible. We cannot ban “mixers” generally – we can only ban “mixers A, B and C.” In some sense, this is akin to banning specific means of murder rather than simply banning murder, no matter the means.

Then they introduce the time element inherent in human regulation:

Second, and more importantly, we cannot enforce even these more straightforward rules reliably. Consider these steps:

Deploy a new, confusingly-coded, “mixer” labeled X

Send funds to the mixer X

Withdraw from the mixer X and feed into the mixer A

Withdraw from the mixer A and feed into the mixer X

Withdraw from the mixer X and spend freely

This procedure works because we cannot identify arbitrary mixers, so we are free to deploy and then use them before they get put on the banned list. As a result, the regulator cannot even ban all interaction with enumerated mixers – it can only reliably ban some forms of interaction. This result is a severe limit on regulatory power.

If we consider that compliance exists in an automated form, operating on publicly available data in real-time, anyone accepting that final transfer must operate in a compliant fashion. If, instead, the plan is to decide these things later based on non-mechanical analysis, we are simply operating a conventional legal system with some more computers involved. Concretely, if that last transfer can be ruled illegal after the fact, it was never an automated financial system.

This result generalizes to services other than mixers. Rice's Theorem means it isn't possible to ban a class of services, and banning individual services identified by humans will always be behind the curve.

The authors illustrate the application of their result with real-world examples of regulatory failure:

These case studies-The DAO, Beanstalk Finance, Compound, Terra/LUNA, and MakerDAO-collectively illustrate the practical manifestations of our paper’s theoretical findings. Each example demonstrates a different facet of the challenges in implementing reliable, automated compliance mechanisms in decentralized, Turing- complete systems. From governance attacks to stablecoin collapses and liquidation issues, these incidents underscore the impossibility of guaranteeing specific regulatory outcomes without compromising system flexibility or introducing external interventions.

The authors argue that there are two ways to construct a system that does allow automated regulation, by making it permissioned rather than permissionless, or by enforcing a non-Turing-complete language for the "smart contracts". In practice many cryptocurrencies are permissioned — in Decentralized Systems Aren't I pointed out that:

The fact that the coins ranked 3, 6 and 7 by "market cap" don't even claim to be decentralized shows that decentralization is irrelevant to cryptocurrency users.

There are many examples of cryptocurrency systems that claim to be decentralized but are actually permissioned. Patrick Tan described one in Binance Built a Blockchain, Except it Didn’t:

For all its claims of promoting decentralization, Binance runs two “blockchains” that are not just highly centralized, but regularly alter history, undermining one of the core tenets of the blockchain — immutability.

The authors' example of a non-Turing-complete language is:

Consider a scripting language where we cannot have variables. A simple “splitting the tab” contract might look like:
This is dangerous if we have a regulation that certain addresses cannot be paid. The issues raised surrounding the DAO hack, discussed above, apply here. But what if the only way to transfer a token is to call SendTo and that function looks like:
There is no issue if a function only calls RealSendTo from SendTo. In such cases, the regulator’s responsibility is to maintain the BannedList, and the system is permissioned.

Their example of a system that is not-Turing-complete is not useful, because it requires BannedList to be a constant. As they point out, if the system is to be useful, BannedList must be a variable that is updated by the regulator, and thus the system is permissioned. It may well be that, because BannedList is a variable, that the system is Turing-complete after all. I can't do the analysis to determine if this is the case, but it is known that even a small number of variables makes a system Turing-complete.

Thus the paper is somewhat misleading, in the sense that it reads as if regulated systems can be either permissioned or not-Turing-complete, but it fails to provide an example of a system that is permissionless and not-Turing-complete. The example that looks as if it is going to be not-Turing-complete but permissionless seems to be permissioned and not-Turing-complete.

I would argue somewhat differently:

Charoenwong et al show that a permissionless, Turing-complete system cannot be regulated.
In the real world no-one is going to cripple their system by making it not-Turing-complete.
Even if a not-Turing-complete system could be built it isn't clear that it would be useful.
In essence, the regulatory act of enforcing that a system is, and remains, not-Turing-complete is permissioning the system.
Thus in practice permissionless systems cannot be regulated.

JPMorgan

While Charoenwong et al's paper is of theoretical interest, in the two years since their initial post it has been overtaken by events. Craig Coben asks Has Michael Saylor’s ‘infinite money glitch’ run into a hitch?:

Reflexivity has been MicroStrategy’s secret sauce. The company prints equity or equity-linked securities to buy bitcoin, boosting bitcoin’s price, which in turn inflates MicroStrategy valuation. This allows it to issue more stock and repeat the cycle.

This MonoStrategy has enabled MicroStrategy to acquire 2.25 per cent of all bitcoin in existence, a hoard worth around $46bn at current prices. The company trades at nearly double the value of its underlying bitcoin holdings, a testament to belief of some investors in Michael Saylor’s project.
...
As long as the stock trades at a premium to its bitcoin holdings, the company can keep issuing novel securities and finding new buyers. Meanwhile, with MicroStrategy comprising a sizeable portion of crypto inflows (28 per cent in 2024, according to JPMorgan), it has helped sustain bitcoin’s ascent.

Both our co-Presidents are heavily invested in pumping cryptocurrencies. The new administration is planning to solve the Greater Fool Supply-Chain Crisis by turning the Federal government into the greater fool of last resort by establishing the strategic Bitcoin reserve. What better greater fool than one who can print his own money? The result is that the S.E.C. Moves to Scale Back Its Crypto Enforcement Efforts, as Matthew Goldstein, Eric Lipton and David Yaffe-Bellany report:

The Securities and Exchange Commission is moving to scale back a special unit of more than 50 lawyers and staff members that had been dedicated to bringing crypto enforcement actions, five people with knowledge of the matter said.

Source

Even if it were possible to regulate real-world cryptocurrencies, there no longer appears to be any political support for doing so. And thus Michael P. Regan reports on A Sunday Night Flash Crash in the ‘Most Insane Casino Ever Created’:

a major flash crash in Ether as leveraged positions were liquidated served as a stark reminder of how digital-asset markets still lack almost all of the guardrails installed on traditional markets over the years due to various misadventures that hurt investors. The second-largest token was down a bit as traditional markets opened for trading Monday morning in Asia, reacting to concerns about US tariffs against Canada and Mexico. Then in a matter of minutes its losses extended to about 27%, before quickly recovering.
...
There’s no indication that any of that is likely to change anytime soon. So Sunday night’s price action in Ether – and similar dives in a slew of other altcoins and memecoins – serves as a reminder that for better or worse this asset class still exists far outside of the padded walls of traditional markets, regardless of how much tradfi embraces it.

Paul Evan Peters Award

2025-01-31T10:00:00.000-08:00

Year	Awardee
2024	Tony Hey
2022	Paul Courant
2020	Francine Berman
2017	Herbert Van de Sompel
2014	Donald A.B. Lindberg
2011	Christine L. Borgman
2008	Daniel E. Atkins
2006	Paul Ginsparg
2004	Brewster Kahle
2002	Vinton Gray Cerf
2000	Tim Berners-Lee

It has just been announced that at the Spring 2025 Membership Meeting of the Coalition for Networked Information in Milwaukee, WI April 7^th and 8^th, Vicky and I are to receive the Paul Evan Peters Award. The press release announcing the award is here.

Vicky and I are honored and astonished by this award. Honored because it is the premiere award in the field, and astonished because we left the field more than seven years ago to take up our new full-time career as grandparents. We are all the more astonished because we are not even eligible for the award; the rules clearly state that the "award will be granted to an individual".

You can tell this is an extraordinary honor from the list of previous awardees, and the fact that it is the first time it has been awarded in successive years. Vicky and I are extremely grateful to the Association of Research Libraries, CNI and EDUCAUSE, who sponsor the award.

Original Logo

Part of the award is the opportunity to make an extended presentation to open the meeting. The text of our talk, entitled Lessons From LOCKSS, with links to the sources and information that appeared on slides but was not spoken, should appear here on April 7^th.

The work that the award recognizes was not ours alone, but the result of a decades-long effort by the entire LOCKSS team. It was made possible by support from the LOCKSS community and many others, including Michael Lesk then at NSF, Donald Waters then at the Mellon Foundation, the late Karen Hunter at Elsevier, Stanford's Michael Keller and CNI's Cliff Lynch.

A Prophet Of The Web

2025-01-16T08:00:00.000-08:00

While doing the research for a future talk, I came across an obscure but impressively prophetic report entitled Accessibility and Integrity of Networked Information Collections that Cliff Lynch wrote for the federal Office of Technology Assessment in 1993, 32 years ago. I say "obscure" because it doesn't appear in Lynch's pre-1997 bibliography.

To give you some idea of the context in which it was written, unless you are over 70, it was more than half your life ago when in November 1989 Tim Berners-Lee's browser first accessed a page from his Web server. It was only about the same time that the first commercial, as opposed to research, Internet Service Providers started with the ARPANET being decommissioned the next year. Two years later, in December of 1991, the Stanford Linear Accelerator Center put up the first US Web page. In 1992 Tim Berners-Lee codified and extended the HTTP protocol he had earlier implemented. It would be another two years before Netscape became the first browser to support HTTPS. It would be two years after that before the ITEF approved HTTP/1.0 in RFC 1945. As you can see, Lynch was writing among the birth-pangs of the Web.

Although Lynch was insufficiently pessimistic, he got a lot of things exactly right. Below the fold I provide four out of many examples.

Page numbers refer to the PDF, not to the original. Block quotes without a link are from the report.

Disinformation

Page 66

When discussing the "strong bias in the Internet user community to prefer free information sources" he was, alas, prescient although it took more than "a few years":

The ultimate result a few years hence — and it may not be a bad or inappropriate response, given the reality of the situation — may be a perception of the Internet and much of the information accessible through it as the "net of a million lies", following science fiction author Vernor Vinge's vision of an interstellar information network characterized by the continual release of information (which may or may not be true, and where the reader often has no means of telling whether the information is accurate) by a variety of organizations for obscure and sometimes evil reasons.

The Vernor Vinge reference is to A Fire Upon the Deep:

In the novel, the Net is depicted as working much like the Usenet network in the early 1990s, with transcripts of messages containing header and footer information as one would find in such forums.

The downsides of a social medium to which anyone can post without moderation were familiar to anyone who was online in the days of the Usenet:

Usenet is culturally and historically significant in the networked world, having given rise to, or popularized, many widely recognized concepts and terms such as "FAQ", "flame", sockpuppet, and "spam".
...
Likewise, many conflicts which later spread to the rest of the Internet, such as the ongoing difficulties over spamming, began on Usenet.:

"Usenet is like a herd of performing elephants with diarrhea. Massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it."

— Gene Spafford, 1992

Earlier in the report Lynch had written (Page 23):

Access to electronic information is of questionable value if the integrity of that information is seriously compromised; indeed, access to inaccurate information, or even deliberate misinformation, may be worse than no access at all, particularly for the naive user who is not inclined to question the information that the new electronic infrastructure is offering.

This resonates as the wildfires rage in Los Angeles.

Information Doesn't Want To Be Free

Although Tim Berners-Lee's initial HTTP specification included the status code 402 Payment Required:

The parameter to this message gives a specification of charging schemes acceptable. The client may retry the request with a suitable ChargeTo header.

the Web in 1993 lacked paywalls. But Lynch could see them coming (Page 22):

There is a tendency to incorrectly equate access to the network with access to information; part of this is a legacy from the early focus on communications infrastructure rather than network content. Another part is the fact that traditionally the vast bulk of information on the Internet has been publicly accessible if one could simply obtain access to the Internet itself, figure out how to use it, and figure out where to locate the information you wanted. As proprietary information becomes accessible on the Internet on a large scale, this will change drastically. In my view, access to the network will become commonplace over the next decade or so, much as access to the public switched telephone network is relatively ubiquitous today. But in the new "information age" information will not necessarily be readily accessible or affordable;

The current RFC 9110 states:

The 402 (Payment Required) status code is reserved for future use.

Instead today's Web is infested with paywalls, each with their own idiosyncratic user interface, infrastructure, and risks.

The Death Of "First Sale"

Lynch understood the highly consequential nature of the change in the business model of paid information access from purchasing a copy to renting access to the publisher's copy; from a legal framework of copyright and the "first sale" doctrine, to one of copyright and contract law (Page 30):

Now, consider a library acquiring information in an electronic format. Such information is almost never, today, sold to a library (under the doctrine of first sale); rather, it is licensed to the library that acquires it, with the terms under which the acquiring library can utilize the information defined by a contract typically far more restrictive than copyright law. The licensing contract typically includes statements that define the user community permitted to utilize the electronic information as well as terms that define the specific uses that this user community may make of the licensed electronic information. These terms typically do not reflect any consideration of public policy decisions such as fair use, and in fact the licensing organization may well be liable for what its patrons do with the licensed information.

The power imbalance between publishers and their customers is of long standing, and it especially affects the academic literature. In 1989 the Association of Research Libraries published Report of the ARL Serials Prices Project:

The ARL Serials Initiative forms part of a special campaign mounted by librarians in the 1980s against the high cost of serials subscriptions. This is not the first time that libraries have suffered from high serial prices. For example, in 1927 the Association of American Universities reported that:

"Librarians are suffering because of the increasing volume of publications and rapidly rising prices. Of special concern is the much larger number of periodicals that are available and that members of the faculty consider essential to the successful conduct of their work. Many instances were found in which science departments were obligated to use all of their allotment for library purposes to purchase their periodical literature which was regarded as necessary for the work of the department"

The oligopoly rents extracted by academic publishers have been a problem for close on a century, if not longer! Lynch's analysis of the effects of the Web's amplification of this power imbalance is wide-ranging, including (Page 31):

Very few contracts with publishers today are perpetual licenses; rather, they are licenses for a fixed period of time, with terms subject to renegotiation when that time period expires. Libraries typically have no controls on price increase when the license is renewed; thus, rather than considering a traditional collection development decision about whether to renew a given subscription in light of recent price increases, they face the decision as to whether to lose all existing material that is part of the subscription as well as future material if they choose not to commit funds to cover the publisher's price increase at renewal time.

Thus destroying libraries' traditional role as stewards of information for future readers. And (Page 30):

Of equal importance, the contracts typically do not recognize activities such as interlibrary loan, and prohibit the library licensing the information from making it available outside of that library's immediate user community. This destroys the current cost-sharing structure that has been put in place among libraries through the existing interlibrary loan system, and makes each library (or, perhaps, the patrons of that library) responsible for the acquisitions cost of any material that is to be supplied to those patrons in electronic form. The implications of this shift from copyright law and the doctrine of first sale to contract law (and very restrictive contract terms) is potentially devastating to the library community and to the ability of library patrons to obtain access to electronic information — in particular, it dissolves the historical linkage by which public libraries can provide access to information that is primarily held by research libraries to individuals desiring access to this information. There is also a great irony in the move to licensing in the context of computer communications networks — while these networks promise to largely eliminate the accidents of geography as an organizing principle for inter-institutional cooperation and to usher in a new era of cooperation among geographically dispersed organizations, the shift to licensing essentially means that each library contracting with a publisher or other information provider becomes as isolated, insular organization that cannot share its resources with any other organization on the network.

Surveillance Capitalism

Lynch also foresaw the start of "surveillance capitalism" (Page 60):

we are now seeing considerable use of multi-source data fusion: the matching and aggregation of credit, consumer, employment, medical and other data about individuals. I expect that we will recapitulate the development of these secondary markets in customer behavior histories for information seeking in the 1990s; we will also see information-seeking consumer histories integrated with a wide range of other sources of data on individual behavior.

The ability to accurately, cheaply and easily count the amount of use that an electronic information resource receives (file accesses, database queries, viewings of a document, etc.) coupled with the ability to frequently alter prices in a computer-based marketplace (particularly in acquire on demand systems that operate on small units of information such as journal articles or database records, but even, to a lesser extent, by renegotiating license agreements annually) may give rise to a number of radical changes. These potentials are threatening for all involved.

He described search-based advertising (Page 61)

The ability to collect not only information on what is being sought out or used but also who is doing the seeking or using is potentially very valuable information that could readily be resold, since it can be used both for market analysis (who is buying what) and also for directed marketing (people who fit a certain interest profile, as defined by their information access decisions, would likely also be interested in new product X or special offer Y). While such usage (without the informed consent of the recipient of the advertising) may well offend strong advocates of privacy, in many cases the consumers are actually quite grateful to hear of new products that closely match their interests. And libraries and similar institutions, strapped for revenue, may have to recognize that usage data can be a valuable potential revenue source, no matter how unattractive they find collecting, repackaging and reselling this information.

Of course, it wasn't the libraries but Google, spawned from the Stanford Digital Library Project, which ended up collecting the information and monetizing it. And the power imbalance between publishers and readers meant that the reality of tracking was hidden (Page 63):

when one is accessing (anonymously or otherwise) a public-access information service, it is unclear what to expect, and in fact at present there is no way to even learn what the policy of the information service provider is.

Storage Roundup

2025-01-07T08:00:00.000-08:00

It is time for another roundup of topics in storage that have caught my eye recently. Below the fold I discuss the possible ending of the HAMR saga and various developments in archival storage technology.

Heat-Assisted Magnetic Recording

Unless you have been tracking storage technology for many years, it is hard to appreciate how long the timescales are. My go-to example for communicating this is Seagate's development of HAMR.

Seagate first demonstrated HAMR in 2002. In 2008 they published this graph, predicting HAMR would supplant Perpendicular Magnetic Recording (PMR) starting in 2009.

I first wrote skeptically about projections of HAMR's deployment twelve years ago. Seagate had just demonstrated HAMR at a terabit per square inch and predicted market entry in 2014.

I wrote again in 2013. In 2015 I wrote more about it. Then in 2016 I wrote about it again.

Seagate's 2018 HAMR roadmap

In 2018 I wrote about Chris Mellor's Seagate HAMRs out a roadmap for future hard drive recording tech::

Seagate has set a course to deliver a 48TB disk drive in 2023 using its HAMR (heat-assisted magnetic recording) technology, doubling areal density every 30 months, meaning 100TB could be possible by 2025/26. ... Seagate will introduce its first HAMR drives in 2020. ... a 20TB+ drive will be rolled out in 2020.

So in a decade the technology had gone from next year to the year after next. The year after next Jim Slater wrote HAMR don’t hurt ’em—laser-assisted hard drives are coming in 2020:

Seagate has been trialing 16TB HAMR drives with select customers for more than a year and claims that the trials have proved that its HAMR drives are "plug and play replacements" for traditional CMR drives, requiring no special care and having no particular poor use cases compared to the drives we're all used to.

But no, it would be another four years before we saw the first signs of HAMR drives in the market. In December 2024 Matthew Connatser reported that Seagate launches 32TB Exos M hard drive based on HAMR technology – Mozaic 3+ drives are the world’s first generally available HAMR HDDs:

Seagate’s biggest-ever hard drive is finally here, coming with 32TB of capacity courtesy of the company’s new HAMR technology (via Expreview).

It has almost been a year since Seagate said it had finally made a hard drive based on heat-assisted magnetic recording (HAMR) technology using its new Mozaic 3+ platform.
...
Exos drives based on Mozaic 3+ were initially released to select customers in small quantities, but now the general release is (nearly) here, thanks to mass production.

Note that the drives that are "(nearly) here" are still not available from Amazon, although they are featured on Seagate's web site. Kevin Purdy writes:

Drives based on Seagate's Mozaic 3+ platform, in standard drive sizes, will soon arrive with wider availability than its initial test batches. The driver maker put in a financial filing earlier this month (PDF) that it had completed qualification testing with several large-volume customers, including "a leading cloud service provider," akin to Amazon Web Services, Google Cloud, or the like. Volume shipments are likely soon to follow.

There is no price yet, nor promise of delivery, but you can do some wishful thinking on the product page for the Exos M, where 30 and 32TB capacities are offered. That's 3TB per platter, and up to three times the efficiency per terabyte compared to "typical drives," according to Seagate.

More indications that volume shipments could happen "next year" comes from Chris Mellor's WD’s HAMR switch could be closer than we think:

Intevac has said there is strong interest in its HAMR disk drive platter and head production machinery from a second customer, which could indicate that Western Digital is now involved in HAMR disk developments following Seagate’s move into volume production.

Intevac supplies its 200 Lean thin-film processing machines to hard disk drive media manufacturers, such as Seagate, Showa Denko and Western Digital. It claims more than 65 percent of the world’s HDD production relies on its machinery. The Lean 200 is used to manufacture recording media, disk drive platters, for current perpendicular magnetic recording (PMR) disks.

Intevac’s main customer for HAMR-capable 200 Lean machines is Seagate, which first embarked on its HAMR development in the early 2000s. It is only this year that a prominent cloud service provider has certified Seagate’s Mozaic 3 HAMR drives for general use, more than 20 years after development first started. The lengthy development period has been ascribed to solving difficulties in producing drives with high reliability from high yield manufacturing processes, and Intevac will have been closely involved in ensuring that its 200 Lean machines played their part in this.

Archival Media

Maybe 2025 will be the year I can finally bring my 12-year-long series about HAMR shipment schedules to a close, 26 years after Seagate started work on the technology. Why have I been HAMR-ing on Seagate all these years, and again now? Not to denigrate Seagate's engineering. Getting a HAMR drive into volume production that meets both the incredibly demanding standards for storage media reliability and performance, and the manufacturing yields needed for profit, is an extraordinarily difficult feat. It is not a surprise that it took a couple of decades.

My criticisms have been aimed at the storage industry's marketing and PR, which hypes developments that are still in the lab as if they are going to solve customers' problems "next year". And at the technology press, which took far too long to start expressing skepticism. Seagate's marketing eventually lost all credibility, with their predictions about HAMR becoming an industry joke.

The situation is far worse when it comes to archival media. The canonical article about some development in the lab starts with the famous IDC graph projecting the amount of data that will be generated in the future. It goes on to describe the density some research team achieved by writing say a gigabyte into their favorite medium in the lab. This conveys four false impressions:

That there is some possibility they could scale their process up to a meaningful fraction of IDC's projected demand, or even to the microscopic fraction of the projected demand that makes sense to archive. There is no such possibility. Archival media is a much smaller market than regular media.
That the researcher's favorite medium could make it into the market in the timescale of IDC's projections. As we see with HAMR, where research started twenty-six years ago, it won't. Research on the silica medium started fifteen years ago. Research on the DNA medium started thirty-six years ago. Neither is within five years of market entry.
That even if the researcher's favorite medium did make it into the market it would be a product that consumers could use. As Kestutis Patiejunas figured out more than a decade ago, the only way to make the economics of archival storage work is to do it at data-center scale and harvest the synergies that come from not needing data-center power, cooling, staffing, etc.
That anyone either cares or even knows what medium their archived data lives on. Only the hyper-scalers do. Consumers believe their data is safe in the cloud. If anyone really cares about archiving they use a service such as Glacier, when they definitely have no idea what medium is being used.

Consumers already have an affordable, durable archival medium. As I have shown:

Surprisingly, with no special storage precautions, generic low-cost media, and consumer drives, I'm getting good data from CD-Rs more than 20 years old, and from DVD-Rs nearly 18 years old.

The market for DVD-R media and drives is gradually dying because they have been supplanted in the non-archival space by streaming, an illustration that consumers really don't care about archiving their data!

DNA Storage

In 2018's DNA's Niche in the Storage Market I imagined myself as the product marketing guy for an attempt to build a rack-scale DNA storage system, and concluded:

Engineers, your challenge is to increase the speed of synthesis by a factor of a quarter of a trillion, while reducing the cost by a factor of fifty trillion, in less than 10 years while spending no more than $24M/yr.

The only viable market for DNA storage is the data-center, and the two critical parameters are still the write bandwidth and the write cost. As far as I'm aware despite the considerable progress in the last 6 years both parameters are still many orders of magnitude short of what a system would have needed back then to enter the market. Worse, the last six years of data center technology development have increased the need for write bandwidth and reduced the target cost. DNA storage is in a Red Queen's Race and it is a long way behind.

Nevertheless, DNA's long-term potential as an archival storage medium justifies continued research. Among recent publications is Parallel molecular data storage by printing epigenetic bits on DNA by Cheng Zhang et al, which avoids the need to synthesize strands of DNA by attaching the bits to prexisting strands. In principle this can be done in parallel. As is traditional, they start by asserting:

The markedly expanding global data-sphere has posed an imminent challenge on large-scale data storage and an urgent need for better storage materials. Inspired by the way genetic information is preserved in nature, DNA has been recently considered a promising biomaterial for digital data storage owing to its extraordinary storage density and durability.

The paper attracted comment from, among others, The Register, Ars Technica and Nature. In each case the commentary included some skepticism. Here are Carina Imburgia and Jeff Nivala from the University of Washington team in Nature:

However, there are still challenges to overcome. For example, epigenetic marks such as methyl groups are not copied by the standard PCR techniques used to replicate DNA, necessitating a more complex strategy to preserve epi-bit information when copying DNA data. The long-term behaviour of the methyl marks (such as their stability) in various conditions is also an open question that requires further study.

Another challenge is that many applications require random access memory (RAM), which enables subsets of data to be retrieved and read from a database. However, in the epi-bit system, the entire database would need to be sequenced to access any subset of the files, which would be inefficient using nanopore sequencing. Moreover, the overall cost of the new system exceeds that of conventional DNA data storage and of digital storage systems, limiting immediate practical applications;

You have to read a long way into the paper to find that:

we stored 269,337 bits including the image of a tiger rubbing from the Han dynasty in ancient China and the coloured picture of a panda ... An automatic liquid handling platform was used to typeset large-scale data at a speed of approximately 40 bits s⁻¹

This is interesting research but the skepticism in the commentaries doesn't exactly convey the difficulty and the time needed to scale from writing less than 40KB in a bit under 2 hours, to the petabyte/month rates (about 2.8TB every 2 hours) Facebook was writing a decade ago. This would be a speed-up of nearly 11 orders of magnitude to compete with decade-old technology.

Diamonds

Chinese boffins find way to use diamonds as super-dense and durable storage medium by Laura Dobberstein reports that:

The research, published in Nature Photonics, highlights that the breakthrough extends beyond density. It is said to offer significant improvements in write times – as little as 200 femtoseconds – and lives up to the promise that "a diamond is forever" by offering millions of years of allegedly maintenance-free storage. Diamonds are highly stable by nature and the the authors have claimed their medium could protect data for 100 years even if kept at 200°C.

High-speed readout is demonstrated with a fidelity of over 99 percent, according to the boffins.

Scientists have been eyeing diamonds as storage devices for a while. Researchers at City College of New York in 2016 claimed to be the first group to demonstrate the viability of using diamond as a platform for superdense memory storage.

These researchers, like so many others in the field, fail to understand that the key to success in archival storage is reducing total system cost. Long-lived but expensive media like diamonds are thus counter-productive.

Project Silica

I wrote about Microsoft's Project Silica last March, in Microsoft's Archival Storage Research. The more I think about this technology, the more I think it probably has the best chance of impacting the market among all the rival archival storage technologies:

The media is very cheap, so the effect of Kryder's Law economics driving media replacement and thus its economic rather than technical lifetime is minimal.
The media is quasi-immortal and survives benign neglect, so opex once written is minimal.
The media is write-once, and the write and read heads are physically separate, so the data cannot be encrypted or erased by malware. The long read latency makes exfiltrating large amounts of data hard.
The robotics are simple and highly redundant. Any of the shuttles can reach any of the platters. They should be much less troublesome than tape library robotics because, unlike tape, a robot failure only renders a small fraction of the library inaccessible and is easily repaired.
All the technologies needed are in the market now, the only breakthroughs needed are economic, not technological.
The team has worked on improving the write bandwidth which is a critical issue for archival storage at scale. They can currently write hundreds of megabytes a second.
Like Facebook's archival storage technologies, Project Silica enjoys the synergies of data center scale without needing full data center environmental and power resources.
Like Facebook's technologies, Project Silica has an in-house customer, Azure's archival storage, with a need for a product like this.

The expensive part of the system is the write head, because it uses costly femtosecond lasers. The eventual system's economics will depend upon the progress made in cost-reducing the lasers.

Self-Own

2024-12-31T08:00:00.000-08:00

Credit: XKCD

The Cambridge dictionary defines the verb to bullshit as:

a rude word meaning to try to persuade someone or make them admire you by saying things that are not true

The essence of successful bullshit is that it should be both plausible and presented authoritatively. Bullshitters are always tempted to buttress the appearance of authority by including actual evidence rather than just their interpretation of the evidence, but this is often a fatal mistake. Below the fold I discuss a classic example from MAGA's campaign to demonize immigrants.

@GabeGuidarini tweeted:

𝐓𝐡𝐢𝐬 𝐢𝐬 𝐭𝐡𝐞 𝐦𝐨𝐬𝐭 𝐢𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭 𝐩𝐨𝐬𝐭 𝐲𝐨𝐮’𝐥𝐥 𝐞𝐯𝐞𝐫 𝐫𝐞𝐚𝐝 𝐨𝐧 𝐦𝐚𝐬𝐬 𝐢𝐦𝐦𝐢𝐠𝐫𝐚𝐭𝐢𝐨𝐧 𝐢𝐧𝐭𝐨 𝐭𝐡𝐞 𝐔𝐧𝐢𝐭𝐞𝐝 𝐒𝐭𝐚𝐭𝐞𝐬:

In 1924, President Calvin Coolidge signed the Johnson-Reed Immigration Act, which halted mass immigration into the United States.

The act completely stopped immigration from Asia, and set strict quotas on immigration from other places including Europe.

Wages began to dramatically increase.

By the 1950s, America reached the peak of its economic and industrial might. Economic inequality was low. Large businesses and workers alike were prospering.

Then, in 1965 the Hart-Cellar Immigration Act was signed by President LBJ, opening the floodgates and allowing for massive unchecked immigration, especially from the global south.

Immediately, income growth and wage growth for the bottom 90% of earners came to an abrupt halt.

Soon after, the incomes for the top 1% of earners skyrocketed, leading to the massive wealth inequality in America we see today.

The political left attributes this to simply a lack of proper taxation and regulation, but that’s mainly not the case.

Mass immigration has allowed for modern scab labor to become the norm, allowing large companies to pay lower wages for foreign migrants who demand less.

The loser? America’s middle class, which has been deteriorating for more than half a century now.

When you see wealthy technocrats argue in favor of mass “skilled” immigration, keep this in mind.

Who is GabeGuidarini?

Ohio Field Rep @TPAction_. Comms @OhioCRs. President @UDRepublicans. Fmr Acting President @uscollegegop.

He is a Republican operative. His chain of causation sounds convincing, doesn't it?

But he made the bullshitter's fatal mistake. He included this graph. Lets annotate it. First we have Guidarini's claim that:

In 1924, President Calvin Coolidge signed the Johnson-Reed Immigration Act, which halted mass immigration into the United States.

The act completely stopped immigration from Asia, and set strict quotas on immigration from other places including Europe.

Wages began to dramatically increase.

Lets put 1924 on the graph.

Whose wages had the dramatic increase in 1924? Right, the 1%. Whose wages were flat from 1924 until the Great Depression? Right, the bottom 90%. Guidarini is just wrong. The dramatic increase in the wages of the bottom 90% happened in 1940, 16 years later. Can you think of a cause for the dramatic increase in 1940? It surely wasn't the Johnson-Reed Immigration Act of 1924.

Second, we have Guidarini's claim that:

By the 1950s, America reached the peak of its economic and industrial might.

If America's "economic and industrial might" peaked in the 50s, why did the incomes of the bottom 90% continue rising until 1973?

Third we have Guidarini's claim that:

Then, in 1965 the Hart-Cellar Immigration Act was signed by President LBJ, opening the floodgates and allowing for massive unchecked immigration, especially from the global south.

Immediately, income growth and wage growth for the bottom 90% of earners came to an abrupt halt.

Lets put 1965 on the graph.

Can you see the income growth for the bottom 90% come to an immediate halt? No, wages continued rising as they had been since 1940 until 1973, another 8 years. Did the Hart-Cellar Immigration Act of 1965 cause wages to flatten in 1973? Whose income flat-lined after 1965 for a decade and a half? Right, the 1%.

Lastly, we have Guidarini's claim that soon after 1965:

the incomes for the top 1% of earners skyrocketed, leading to the massive wealth inequality in America we see today.

Can you see when the "incomes for the top 1% of earners skyrocketed"? That's right, it was in 1985, so "soon" means 20 years! And can you guess who was President of the US, and from what party, then? Right, it was a certain Ronald Reagan, a Republican.

Cherry-picking

2024-12-17T08:00:00.000-08:00

Source

Via Barry Ritholtz we find this infographic entitled Top Performing S&P 500 Stocks showing the best total return over the past 5, 10, 15 and 20 years. Ritholtz sourced it from Ranked: The Top Performing S&P 500 Stocks in the Last Two Decades By Marcus Lu with graphic design by Miranda Smith.

In each case, Nvidia is the best performing stock, and it is the only stock to appear in all four periods. Sounds great, doesn't it? Why wouldn't you just hold NVDA all the time and be guaranteed to beat the market?

But follow me below the fold for more detail from someone who has been long NVDA for more than three decades..

Source

Alas, this infographic is deeply misleading because they cherry-picked their data. Nvidia's stock price is extraordinarily volatile. The log plot of NVDA shows that on average over its history every three years it suffers a drop of between 45% and 80%. Fortunately, over the same time it has had much larger rises. Thus when discussing the return for being long NVDA for a period the start and end dates matter a lot.

Dec	Close	Dec	Close	NVDA	S&P
2004	0.19	2009	0.38	+100%	-9.1%
2009	0.38	2014	0.48	+26%	+85.8%
2014	0.48	2019	5.91	+1131%	+61.7%
2019	5.91	2024	134.25	+22715%	+87.6%

This table shows the performance of NVDA and the S&P 500 over each of the 5-year periods in the infographic. Over three of the 5-year periods NVDA out-performed the S&P 500, and over the remaining one it under-performed. By choosing appropriate starting points it is easy to find other 5-year periods when NVDA under-performed the S&P 500. For example, from 8/2007 to 8/2012 NVDA returned -63.7% but the S&P returned -5.6%. Or from 11/2001 to 11/2006 NVDA returned +10.7% but the S&P returned +23.5%.

Source

Although the infographic suggests that the huge out-performance started 20 years ago and has been decreasing, this is misleading for two reasons. First, the infographic's numbers are for the whole 20-year period, not for each 5-year period. Second, they are dominated by the rise in the most recent 5-year period. The linear plot of NVDA makes it clear that, had the most recent 5-year period been like any of the others, Nvidia would not have been in the infographic at all.

None of this is financial advice. Nevertheless, Nvidia is a great company, and it is possible to make money in its stock. There are two ways to do it, trading on the stock's volatility, or buying low and being prepared to hold it for many years.

Remember, past performance is no guarantee of future performance. If NVDA were to repeat the past 5-year return over the next 5 years, it would be around $3,100. To maintain its current P/E of 52.85 it would need annual revenue of around $2.6T.

Drones

2024-12-03T08:00:00.035-08:00

[email protected]#/media/File:[email protected]" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;">

Source

In 2017 the Future of Life Institute released a video entitled Slaughterbots. Wikipedia describes it as an:

arms-control advocacy video presenting a dramatized near-future scenario where swarms of inexpensive microdrones use artificial intelligence and facial recognition software to assassinate political opponents based on preprogrammed criteria.

War accelerates technological progress. The war in Ukraine has not yet produced "slaughterbots" but it has greatly accelerated drone technology and taken some giant steps toward them. The most important of these steps is that the cost of precision strike has been reduced by 1-2 orders of magnitude, making it affordable for "non-state actors" and even individuals.

Below the fold I look at drone developments in the war in Ukraine, what is happening with drones and drone defense in the West, and sketch some implications for the future.

In parallel with the military developments I discuss here, there is rapid evolution in civilian drone systems, especially in China, which can be expected to feed into the military space. For example, Arnaud Bertrand describes Pingyin County's low-altitude economy:

The "low-altitude economy" is a big trend in China at the moment. XPeng, one of China's leading EV manufacturers, recently released a low-altitude flying car for instance. Drone deliveries are becoming increasingly common in Chinese cities, and various regions are actively developing low-altitude transportation networks. Shanghai, for instance, plans to establish 400 low-altitude flight routes by 2027.
...
this experiment in Pingyin County might seem small, but it could be the beginning of something truly revolutionary. This is China effectively experimenting around becoming the world's first truly three-dimensional economy, with low-altitude transportation and logistics networks. In a decade or two Chinese cities might have as much activity happening in the sky as on the ground.

Ukraine

The drone systems I am discussing here fall into two classes:

Small systems such as First Person View (FPV) quadcopters, with a range of a few tens of kilometers.
Larger systems navigating autonomously by GPS or other means with a range of hundreds of kilometers but relatively slow speeds of up to 200Kph.

I am excluding faster systems such as short-range missiles like HIMARS and cruise missiles such as Storm Shadow.

Short Range Drones

The battlefield use of small fixed-wing and FPV quadcopter drones in the class of DJI's Mavic and Matrice has transformed both reconnaissance and strike. They cost a few hundred to a few thousand dollars; for comparison a 155mm shell is around $3K.

Reconnaissance

By Mil.ru, CC BY 4.0, Link

Small drones, such as the Russian Orlan-10, provide an eagle's-eye view of the battlefield with real-time video. They have greatly extended the range of battle field surveillance. With zoom lenses they can provide detail at these longer ranges. With GPS they can provide precise target locations. With thermal cameras they can do all this at night.

These changes have had significant tactical effects:

Movement within about 20Km of the front line has become much more dangerous. Night is no longer a good cover for movement. This has made logistical support for positions close to the front more difficult. For example:

That has made some sections of the front lines, for example around Siversk in Luhansk province, practically no-go areas for humans. Drones are now responsible for a majority of battlefield losses, overtaking artillery, according to Ukrainian sources.
Damage assessment has become much easier. Real-time video of strikes can estimate their effects accurately, avoiding wasting ammunition by repeated strikes on targets that have been sufficiently disabled.
The vast amount of video generated, much of it from drones and the rest from cellphones allows selected clips to be posted to social media used for propaganda and for units to fund-raise. Donations are significant, especially on the Russian side for vehicles, and on the Ukrainian side to support drone manufacture:

The Ukrainian state is buying more than it once was, but voluntary foundations still provide at least a third of all drones used by the army.
This cornucopia of video fuels an active open source intelligence community and especially their impressive ability to geolocate. It has become much harder for combatants to hide events from the public.

Strike

Kamikaze FPV drones provide sniper-like precision strike, but at vastly longer range, with much heavier warheads, and the ability to access targets that snipers cannot. For example, social media videos routinely show FPV drones striking targets inside buildings or infantry fighting vehicles (IFVs) after entering through open doors. The warheads in common use include anti-armor explosively formed projectiles — here is a small FPV drone costing a few thousand dollars demolishing a T-90, the most advanced tank Russia has deployed costing perhaps $3-5M. Note the precision with which the FPV drone strikes the joint between the turret and the hull. And here four FPV drones totaling around $10K destroy a $24M Zoopark radar.

Compare this to the US FGM-148 Javelin with a unit cost of around $250K, a range of less than 5Km and weighing 22Kg. Is one Javelin as effective as 50 FPV drones delivering perhaps six times the weight of warhead at more than twice the range?

There are also many FPV strike drones that are not kamikaze. There are "Dragon drones" that drop burning thermite on trenches or disabled armor. The Wild Hornets drone manufacturer claims their Queen Hornets product:

is a versatile platform capable of mining, bombing enemy positions, delivering supplies, transporting drones, firing an assault rifle or grenade launcher, and relaying communications

Then there is the night bomber drone the Ukrainians call "Vampire" and the Russians Baba Yaga:

The drone is a large hexacopter (6-rotor) drone. It is equipped with an infrared (thermal imaging) camera and able to carry up to a 33-pound (15kg) rocket warhead.
...
Baba Yaga was used both as a bomber deploying "mortar-sized munitions" and also as a "mother-ship" drone equipped with a signal repeater, anti-jamming equipment, batteries, and directional antennas. Its ability to function as a signal repeater extended the battery life and range of secondary drones. During the Russian invasion of Ukraine, the drones have been used primarily at night.

@PStyle0ne1 tweeted video of a Baba Yaga destroying a Russian BUK-M1 air defense system hidden in a forest at night with a precise bomb drop. Drones in this class are also highly effective at laying minefields.

Although Baba Yaga carries the same size warhead as a Shahed 131, there is a huge gap in range and speed. But systems are emerging to fill it, such as one in the video tweeted by @Bricktop_NAFO that converts between fixed-wing and quadcopter in flight.

There are now also specialized drone motherships that carry 4-6 FPV drones and relay their communications, greatly extending their range of action.

ZALA Lancet
Nickel nitride, CC0

On the Russian side the predominant strike drone is the Lancet, a propeller-driven fixed-wing drone costing around $20K with a range up to 40Km. It has been fairly effective until recently (see Defense below), but probably less cost-effective than the much cheaper quadcopter FPV drones.

The Achilles heel of all these drones is their dependence upon radio communication for command and control, which can be both detected and jammed. Worse, the command and control transmitter can be located and attacked; drone pilots are a high-priority target for both sides. This tweet shows the Ukrainians using remotely-controlled directional antennas so to minimize the risk of detection, and the loss if attacked.

Thus both sides have started to deploy drones controlled via a fiber optic cable. David Hambling reports on a Ukrainian version in Ukraine Fields Unjammable Fiber Optic FPV Attack Drone:

Khyzak REBOFF is a new Ukrainian FPV kamikaze drone with fiber optic cable for communication instead of radio. This makes it immune to jamming , the most effective protection against small drones.

Russia has deployed similar technology with the Prince Vandal FPV used in action since August, and which has been lethally effective. However, Ukrainian expert Serhii Flash believes this is a Chinese model which the Russian suppliers rebranded and sold to the military (at a 750% markup), and which was not designed for military use.

The video in this tweet shows that the fiber doesn't impair the FPV drone's remarkable agility. And this tweet from the Russian side highlights the high-definition video characteristic of fiber optic drones. This thread discusses the many advantages fiber optics have over radio in drone warfare. However, replacing radio with fiber adds cost and weight, and reduces range.

The other way to prevent detection and neutralize jamming is to remove the need for a remote pilot by having the drone detect potential targets and select the most valuable autonomously, as implemented for example in the UK's "fire-and-forget" Brimstone missile. Brimstone costs around $130K, over an order of magnitude more than these Ukrainian autonomous drones. They are cheap enough to be used in swarms, as shown in this tweet's video.

The US military reports that:

Attaching infrared and visible light sensors to unmanned systems with onboard object detection and tracking will also increase model performance and targeting effects regardless of illumination conditions. The Ukrainian army is already deploying semiautonomous multirotor drones with onboard edge devices to fly munitions into Russian tanks autonomously.

Max Hunter's Ukraine rolls out dozens of AI systems to help its drones hit targets provides more detail:

Ukraine is using dozens of domestically made AI-augmented systems for its drones to reach targets on the battlefield without being piloted, a senior official said, disclosing new details about the race against Russia to harness automation.

Systems that use artificial intelligence allow cheap drones carrying explosives to spot or fly to their targets in areas protected by extensive signal jamming, which has reduced the effectiveness of manually piloted drones.
...
A Ukrainian official told Reuters in July that most first person view units' target strike rate had fallen to 30%-50%, while for new pilots that can be as low as 10%, and that signal jamming was the main problem.

The official predicted that AI-operated first person view drones could achieve hit rates of around 80%.

Source

These systems are already in action. @GrandpaRoy2 tweeted:

The Russians claim they have recovered a Ukrainian FPV with AI target acquisition and terminal guidance.
...
The FPV has a ‘Google Coral AI’ single board computer for machine language development.

That is a $130 add-on. Special Kherson Cat tweeted video of an attack:

The 129th TD Brigade stopped a Russian BTR-82 in Kursk region, and finished it off with FPV with an automatic targeting system

The Odessa Journal's Ukrainians have created an AI for drones that automatically identifies and strikes targets provides more detail:

Our AI system not only recognizes static targets but also performs precise targeting of moving objects at speeds of up to 60 km/h. It can simultaneously track multiple targets and prioritize one for strike
...
Unlike traditional computer vision technologies, this system was trained on a specially created database and can recognize seven types of targets: infantry, cars, minivans, trucks, air defense systems, artillery, tanks, and armored vehicles.
...
The automatic target recognition range varies from 150 to 800 meters depending on the object type, with the maximum targeting range being up to 1000 meters. The system can be installed on any type of FPV kamikaze drones, including multicopters, wings, and airplanes. It is equipped with its own camera, connects to the flight controller and standard VTX, and drone control is done via a standard remote.

According to the interview, the system costs about $50 more than traditional targeting systems.

The Economist's How Ukraine uses cheap AI-guided drones to deadly effect against Russia agrees with this cost estimate:

Yaroslav Azhnyuk, the founder of The Fourth Law, says that his own “autonomy module” is around $50 to $100 per unit for Ukrainian customers buying thousands of units.

The cost-benefit ratio of adding AI is reportedly considerable:

Data from the battlefield suggest that the hit rate for these AI-guided drones is currently above 80%. That is higher than the rate of manually piloted drones. As important, the training burden declines dramatically. Mr Liscovich notes that, although there are now more highly experienced drone pilots, some with thousands of hours of flying time, the average quality of Ukrainian personnel has fallen over time as less motivated people are conscripted. “We can train an operator within 30 minutes and the quality of the engagement doesn’t depend on their piloting skills,”

With feedback from use in combat, the performance of these systems is improving rapidly:

Rudimentary object-recognition software has been in use, on both sides, for over a year. But it is getting better. Lorenz Meier of Auterion, a Switzerland-based firm, says that between spring and summer his firm’s software, known as Skynode, managed to double the range at which a drone could engage a target, from 500 metres to 1km or so. He says that improvements in the resolution of images captured by drones have since increased that further. The Economist understands that AI systems are sometimes locking onto targets at perhaps double that distance, far beyond the range at which basic jammers could take out the drone.

The innovation cycle is accelerating:

Mr Meier reckons that fewer than a tenth of drones are AI-guided at present. But that number is rising. Ukraine’s innovation cycles are relentless, with feedback loops in some cases down to a few days. The simplicity of FPV drones, which Ukraine builds from off-the-shelf components, can also act as a hindrance: it makes them easy to replicate. One manufacturer says Russian reverse engineering can be as quick as three weeks, but encryption means that the software can be protected from copying.

These autonomous systems are still vulnerable to jamming before they get close enough for the AI to recognize the target. Some idea of the intensity of the electronic warfare in Ukraine can be gained from Charlie Metcalfe's Meet the radio-obsessed civilian shaping Ukraine’s drone defense:

Serhii “Flash” Beskrestnov hates going to the front line. The risks terrify him. “I’m really not happy to do it at all,” he says. But to perform his particular self-appointed role in the Russia-Ukraine war, he believes it’s critical to exchange the relative safety of his suburban home north of the capital for places where the prospect of death is much more immediate. “From Kyiv,” he says, “nobody sees the real situation.”

So about once a month, he drives hundreds of kilometers east in a homemade mobile intelligence center: a black VW van in which stacks of radio hardware connect to an array of antennas on the roof that stand like porcupine quills when in use. Two small devices on the dash monitor for nearby drones. Over several days at a time, Flash studies the skies for Russian radio transmissions and tries to learn about the problems facing troops in the fields and in the trenches.

He is, at least in an unofficial capacity, a spy. But unlike other spies, Flash does not keep his work secret. In fact, he shares the results of these missions with more than 127,000 followers—including many soldiers and government officials—on several public social media channels. Earlier this year, for instance, he described how he had recorded five different Russian reconnaissance drones in a single night—one of which was flying directly above his van.

@GrandpaRoy2 reports that:

Both sides in Ukraine are developing video frequency channels lower than the standard 5.8GHz.

The Russians are using technology from satellite TV dishes to make a 3.3GHz video channel for their fixed-wing reconnaissance drones.

Ukrainian drone units have developed an analog video board for their FPVs at 1.2GHz, to escape interception and monitoring by the Russians.

Here is a thread discussing the hand-held drone detectors snooping on the drones' video channels.

How CRPA works

Since the longer-range drones navigate using GPS or the Russian GLONASS, they are vulnerable to jamming and spoofing. Sophisticated Controlled Reception Pattern Antennas (CRPA) can filter out the jamming signal:

The CRPA antenna exploits the fact that a GPS satellite signal and a ground based jamming signal arrive from different directions. It consists of an array of antennas that measure the direction and time of arrival of different signals

CRPAs can be enhanced with algorithms to defend against spoofing, but they are only partly effective:

Cometa CRPA

Recently Russian Shahed drones have been ‘spoofed’ when their global navigation satellite systems (GNSS) received Ukrainian false coordinates that resulted in them flying back to Russia and Belarus . The “Cometa” GNSS is resistant to jamming, but vulnerable to spoofing.

The Cometa CRPA (Controlled Reception Pattern Antenna) GNSS was upgraded this year to a jam-resistant 8 antenna array, but is obviously still vulnerable to spoofing.

Because both reconnaissance and strike drones are expendable, mass production is critical. The Ukrainian Review reports that:

For 2024-2025, MoD have already contracted 1.8 million drones for almost UAH 147 billion.

Long-range drones, FPV, reconnaissance copters, aircraft-type UAVs, attack copters.

Another 366,940 drones are planned to be delivered by the end of 2024.

Source

One essential component of the short-range drones is their battery. David Hambling reports that Ukraine Shows U.S. How To Beat China In Drone Battery Wars:

Most drone batteries are made in China, which is why the Chinese government was able to pull the plug on battery supplies to U.S. military drone maker Skydio last month.

Skydio CEO Adam Bry reassured customers that they would quickly find an alternate source, but warned that for the next few months Skydio X10 batteries would be rationed — one per drone.
...
While not all cells are created equal, they are essentially commodity products manufactured by the billion. They’re made mainly by big players in the Far East; China dominates but it does not have a monopoly. Other sources are readily available.
...
To build your own drone batteries, you have to source quality cells from a reliable supplier and assemble them into battery packs. And that is exactly what Ukrainian drone maker Wild Hornets has been doing for some time.

A video on social media explains Wild Hornets' process. The building blocks for its battery packs are Samsung 50S, which are optimized for high-power applications and have a respectable 5000 mAH capacity.

FPV drones may be relatively cheap, but a war consumes them at an enormous rate. The technical problems can be solved but the financial ones are harder:

Last month, President Volodymyr Zelenskiy said in a speech his country had produced 1.5 million drones so far this year, far short of an annual manufacturing capacity of over 4 million. All that was lacking is the money to buy them all.

Source

Lessons from Ukraine are being learned world-wide, and innovation proceeds elsewhere. For example, here is video of the Syrian rebels using a rocket-powered fixed-wing FPV drone, or rather a ground-launched FPV glide bomb. The picture is of one downed in action a year ago. Note that the Ukrainian HUR was known to be active in Syria before that.

Longer Range Drones

Shahed 136
Alexpl CC BY 4.0

The typical longer-range drone in Ukraine is a small, propeller-driven fixed-wing aircraft, such as the Iranian Shahed, used by the Russians in large numbers:

The Shahed 131 is 2.6m long, spans 2.2m and has a range of 900Km with a 15Kg warhead.
The Shahed 136 is 3.5m long, spans 3.5m and has a range of 2500Km at 185Kph with a 50Kg warhead.

There are also larger drones up to the size of small light aircraft, which the Ukrainians have used for raids such as the recent 34-drone attack on Moscow.

Marc Champion's Above Ukraine, a Sky Full of Drones Is Changing the War describes one of these larger drones:

In Ukraine, hundreds of new drone manufacturers sprung up to meet demand. One of them, FP (formerly named Fire Point), was founded by a group of Ukrainian movie producers and businessmen and released an unmanned mini plane in the summer of 2023. FP’s drone can fly 1,000 kilometers at up to 205 km per hour, carrying 60 to 100 kilograms (130 to 220 pounds) of explosives. Ukrainian forces have so far used them to strike six airfields, two arms depots and a naval base in Russia, according to the company.

HX-2

As with FPV drones, AI is being applied to longer-range drones. For example, the German company Helsing is apparently planning to ship Ukraine 4000 of its new HX-2 drone:

This kamikaze drone has a range of up to 100 km and weighs about 12 kg and a speed of up to 220 km/h

The main feature of the HX-2 is its integration with advanced machine vision algorithms, which provide autonomous target recognition and their destruction even with active electronic warfare.

This is similar to Brimstone's capabilities but slower, smaller, longer-ranged and much cheaper.

Again, the most important aspect of these developments is that they greatly reduce the cost of precision strike :

Ukraine’s new drones are also cheap. A kit of 10 Fire Point drones plus a control station and transport, including three mobile trailers, costs $580,000. That’s a fraction of the cost of military drones produced by western arms manufacturers, though it’s still a stretch for Ukraine’s military budget. FP says it can produce at least 500 drones per month, but that Ukraine’s defense ministry struggles to buy a fifth of that.

Drone Defense

The essential feature of a successful defense against either the short- or the longer-range drones is that it be cheaper than the attack. This is rapidly decreasing, making cost-effective defense challenging.

Source

One of the effective anti-Shahed systems in Ukraine is pickup trucks mounted with machine guns such as the Browning M2 and improvised thermal sights. The Browning M2 is a remarkable weapon. It dates from World War 1 and:

"It is the primary heavy machine gun of NATO countries and has been used by many other countries as well. U.S. forces have used the M2 longer than any other firearm except the .45 ACP M1911 pistol, which was also designed by John Browning.

This tweet has video of one of them downing a Shahed 136.

Yak-52 and kill

Along similar lines is the 1970s Yak-52 armed with a shotgun which defends Odesa:

The Yak-52, which costs mere hundreds of dollars per flight hour, presents a cost-effective solution against drones valued around $100,000. In a remarkable three-month period starting May, the aircraft reportedly shot down at least a dozen Russian drones, with increasing kill markings documenting its effectiveness,

Source

All drones emit heat, and the sky is generally a fairly even temperature looking up at it so the passive thermal sights are effective. The pickups and the Yak-52 rely on a nationwide passive acoustic sensor system built from cell-phones, using machine learning to identify drones acoustic signatures. It is called Sky Fortress:

Over a field east of Poltava, the sound of a 4-cylinder piston engine rumbles through the quiet of the night. A small electronic box attached to a cell tower records the sound, identifying the audio profile of a Geran-2 loitering munition, and transmits the data for processing. Responding to an alert, two soldiers fire up the engine on their Nissan pickup truck and go speeding down the road to the grid coordinates provided. In position, they scan the horizon in the direction of the attack. They have only a narrow window of opportunity. Tracers arc into the sky from their old but reliable DShK, their efforts rewarded as the drone disappears in a fireball.

Zipline drone

Shaheds and quadcopters are quite noisy, but Zipline, the South San Francisco autonomous drone delivery company, has made remarkable progress in rotor design to reduce the noise signature, as Mark Rober demonstrates in this video.

Inevitably, FPV drones started to be used to counter FPV drones. Here, for example, a kamikaze FPV drone intercepts an enemy FPV drone.

Ukrainian FPV drones have been extremely effective against Russian reconnaissance drones. The video in this tweet documents the loss of 406 assorted Russian reconnaissance drones.

Source

This summer the Ukrainians launched a campaign to use FPV drones to intercept Lancets. As the graph shows, this has caused a remarkable reduction in the number of Lancet strikes.

It isn't necessary to use kamikaze drones in the fighter role. Both sides are now using FPV drones as net launchers, entangling rotors or propellers to bring their targets down.

Source

Artem's The Ukrainian Defense Forces intercepted more than 850 drones of the “fixed-wing” type is a painstaking analysis from open sources:

these models can be further categorized into either ‘reconnaissance’ or ‘strike’, with 11 reconnaissance models and 3 strike models, of which 850 and 20 were intercepted,
...
It should be noted that this study does not include examples and video footage of Russian FPV interceptions; we note that the presently recorded ratio is 870:8 in this regard.

Note that this analysis confirms the effectiveness of the drone interception campaign but shows it started in late August.

Source

When all else fails, the one remaining option is to accept that the drone will reach the target but armor it so that it, or at least the crew, survives to fight on. The Kyiv Post describes the evolution of armored vehicles to counter the drone threat in From Sublime to Ridiculous – Russia’s T-80 ‘Super Turtle Tank’:

In mid-2023, faced with the ever-growing threat from Ukraine’s first-person view (FPV) kamikaze drones, Russian tanks and armored personal carriers (APCs) began being fitted with so-called “cope cages” aimed at limiting the impact of attack.

Initially these took the form of improvised crude grills, frames and screens to protect their turrets and other vulnerable parts. Over time, the cages became bigger, eventually covering most of the vehicle, with reports that anti-drone grills were being fitted to newly built tanks and APCs in the factory. They also started to appear on Israeli tanks moving against Hamas in Gaza.

In early April this year, Russian troops, seemingly concerned that their improvised and even factory-fitted grills weren’t providing adequate protection, began supplementing their cages with armored boxes covering the whole tank

The West

The rapid evolution of drone technology is posing significant problems for Western nations. Drones are already being used to surveil defense assets with impunity, because neither suitable defense systems nor appropriate policies for their use are in place.

Drone Incursions

Shahed drone

Three weeks ago Gordon Lubold, Lara Seligman and Aruna Viswanatha of the Wall Street Journal reported that the Mystery Drones Swarmed a U.S. Military Base for 17 Days. Pentagon Is Stumped:

U.S. Air Force Gen. Mark Kelly wasn’t sure what to make of reports that a suspicious fleet of unidentified aircraft had been flying over Langley Air Force Base on Virginia’s shoreline.

Kelly, a decorated senior commander at the base, got on a squadron rooftop to see for himself. He joined a handful of other officers responsible for a clutch of the nation’s most advanced jet fighters, including F-22 Raptors.

For several nights, military personnel had reported a mysterious breach of restricted airspace over a stretch of land that has one of the largest concentrations of national-security facilities in the U.S. The show usually starts 45 minutes to an hour after sunset, another senior leader told Kelly.

The first drone arrived shortly. Kelly, a career fighter pilot, estimated it was roughly 20 feet long and flying at more than 100 miles an hour, at an altitude of roughly 3,000 to 4,000 feet. Other drones followed, one by one, sounding in the distance like a parade of lawn mowers.

The drones headed south, across Chesapeake Bay, toward Norfolk, Va., and over an area that includes the home base for the Navy’s SEAL Team Six and Naval Station Norfolk, the world’s largest naval port.

Given the incredible lethality of drones in Ukraine, this seems like a problem. The WSJ story provided more details:

Over 17 days, the drones arrived at dusk, flew off and circled back. Some shone small lights, making them look like a constellation moving in the night sky—or a science-fiction movie, Kelly said, “‘Close Encounters at Langley.’” They also were nearly impossible to track, vanishing each night despite a wealth of resources deployed to catch them.

Gen. Glen VanHerck, at the time commander of the U.S. Northern Command and the North American Aerospace Defense Command, said drones had for years been spotted flying around defense installations. But the nightly drone swarms over Langley, he said, were unlike any past incursion.

The swarms were clearly a sophisticated operation:

U.S. officials didn’t believe hobbyists were flying the drones, given the complexity of the operation. The drones flew in a pattern: one or two fixed-wing drones positioned more than 100 feet in the air and smaller quadcopters, the size of 20-pound commercial drones, often below and flying slower. Occasionally, they hovered.

They came from the north around 6 p.m. to traverse the base, which sits on a peninsula at the mouth of Chesapeake Bay, and continued south, beyond the reach of radar. They repeated the pattern and then disappeared, typically by midnight.

Shaheds do sound like lawn movers, fly around 100mph, and have the endurance to fly several circuits of Langley between 6pm and midnight. General Kelly might have over-estimated the length; the fixed-wing drones may have been Shahed 136 equivalents. The map shows the vast area from which a Shahed 136-class drone attack on Langley could be launched.

In Here’s What NORAD’s Commander Just Told Us About The Langley AFB Drone Incursions Howard Altman reports that General Guillot:

offered few specifics about the incursions but talked about some of the steps NORTHCOM is taking in the wake of those incidents.
...
Replying to our question, Guillot told us he did not know if they were tracked back to their recovery point or whether they could have been launched by a vessel off the coast.

The Langley incursions were among more than 600 reported over U.S. military installations since 2022, NORAD stated Tuesday. In the wake of the rash of drone activity over Langley AFB, NORTHCOM was tasked by the Joint Chiefs of Staff to look at drone incursions across the U.S., Guillot said.

But these drones weren't their problem:

“I saw that NORAD’s responsibility for countering UAS was very limited to something that would be an attack of national consequence,” he explained not the small drones as seen over Langley and elsewhere. NORTHCOM, meanwhile, has no responsibility or authority to take action, because the services are charged with securing their facilities.

Altman explains that the Dept. of Defense has had their head in the sand about the threat for years:

As we have frequently reported, there have been unidentified drone incursions for years over U.S. military installations, warning areas, and critical government facilities. Among them were a very similar repeated rashes of claimed drone sightings in Colorado in 2020 in an area where many of America’s ICBMs are based. Another took place at the Palo Verde nuclear facility in Arizona in 2019.

Also in 2019, drones swarmed over a U.S. Navy exercise taking place 100 miles off the coast of California, raising concerns that an adversary was trying to suck up critical intelligence, including very sensitive electronic and signals emissions of America’s most advanced air defense and command and control systems.

More recently, we were the first to write about drone flights over the highly-secure Plant 42 in Palmdale California that were so concerning a temporary flight restriction was issued for the airspace over the facility.

Using FOIA requests, The War Zone's Jospeh Trevithick, Adam Kehoe and MARC Cecotti have revealed similar reports dating back to 2016. The Press Association's Unidentified drones spotted over three UK airbases, US air force confirms makes it clear this isn't just a US problem:

Unidentified drones were spotted over three airbases in Britain used by the US air force (USAF)

“Small unmanned aerial systems” were seen between 20 and 22 November over RAF Lakenheath and RAF Mildenhall in Suffolk, and RAF Feltwell in Norfolk.

And also:

A mystery drone was seen trailing HMS Queen Elizabeth, the British aircraft carrier, as it entered the port of Hamburg, in Germany, on Friday

These incursions are continuing.

In some cases the source of the drone incursions is obvious. Since at least 2017 non-state armed groups (NSAGs), such as Mexican drug cartels, have acquired greatly enhanced strike and reconissance capabilities. For example, Rio Grande Valley Sector Chief Patrol Agent Gloria Chavez testified that:

in the Rio Grande Valley sector of Texas alone, Border Patrol has faced over 10,000 drone incursions and 25,000 drone sightings in just one year.

An academic study of this problem is Narco drones: tracing the evolution of cartel aerial tactics in Mexico’s low-intensity conflicts by Ghaleb Krame, Vlado Vivoda and Amanda Davies,which:

examines the evolution of drone tactics employed by drug cartels in Mexico from 2017 to 2022. The research traces the increasing sophistication of drone technology, payload capacities, and adaptability of cartels in employing airborne drones in low-intensity conflicts.

Note that this research does not include developments driven by the Ukraine war. The rapid cost reductions in drone technology since likely mean much greater use of drones by criminal organizations.

Drone Defense: Systems

General Guillot outlined his approach to drone defense:

“I think there’s certainly a role for fighters. There’s a role for non-kinetic or low-collateral kinetic capabilities that would be at the smaller end. And then I’m a big proponent for advances in directed energy, laser and high power microwave. So those non-kinetic capabilities that bring a kinetic result of bringing down the systems, and I think it has to be layered in today’s today’s threat.”

Source

Indeed, the DoD's Palletized High Energy Laser Program has produced a laser counter-drone system, the BlueHalo LOCUST Laser Weapon System. The WSJ reported on it in this video.

One of the big problems with cheap drones is that you need to shoot them down even more cheaply. LOCUST touts a per-shot cost of just $3, but that isn't the end of the story. Each LOCUST costs $10M. "fewer than a dozen are deployed", all overseas. Each requires a dedicated team from BlueHalo to maintain it because there are no stocks of spare parts. And "the Pentagon has not confirmed that a laser has taken down a drone in combat". It weighs 3400lb and fits in a 7' cube. It takes "as many as 15 seconds to melt a target".

Despite the low per-shot cost, LOCUST isn't a solution to the rash of drone swarms. First, at a unit price of $10M plus a dedicated support team defending the vast number of US defense assets isn't affordable. Second, the problem isn't defending against a single drone, it is defending against a swarm of cheap drones. For example, Ukraine used at least 34 drones in their November 10^th attack on Moscow. Lets guess that Locust can switch targets in 5 seconds, and is 100% lethal in 15 seconds, it would take 11:20 to defeat the swarm. It isn't going to have that long, because at 100mph that is 19 miles. So even if 34 was the biggest feasible swarm (hint: it isn't) defending a high-value asset would need say 5 LOCUST systems at $50M capex plus opex.

Drone Defense: Policy

Neither the pickups with the Brownings nor the cellphones are systems the DoD's supply chain is interested in building. But even if systems like LOCUST could be afforded and deployed, policies need to be established for their use. Joseph Trevithick, Howard Altman and Tyler Rogoway's Lasers, Microwaves, Missiles, Guns Not On The Table For Domestic Drone Defense is a deep dive into the policy issues:

Even if counter-drone policies are clarified and solidified, and authorities get expanded, there are still other concerns at play, especially when it comes to collateral damage. Shooting something down with a gun-based system or a surface-to-air missile carries inherent dangers of projectiles or interceptors, or debris from them, falling onto innocent bystanders below. The aforementioned Centurion C-RAM fires specialized self-destructing ammunition to reduce these risks.

Here, for example, is an Oerlikon Revolver in Ukraine firing air-burst ammunition. But the ammunition isn't the only problem:

The drone, or what is left of it, which could include undetonated explosive warheads or other potentially hazardous payloads, would also fall to Earth in an uncontrolled manner after any such engagement.

The Russians frequently explain burning oil refineries and similar mishaps on debris from successful interceptions of drones. Someone whose house was destroyed by the 50Kg warhead of a shot-down Shahed 136 would have reason to be upset. Deciding on a suitable counter-drone policy is complex:

Maybe even more importantly is that these weapons do fail and they are packed with high explosives and traveling at high speed, which could cause very unpredictable harm to people and property in a large potential impact area. Then there is the idea that you have to know exactly what you are shooting at, and that can be harder than it sounds when dealing with strange objects in the sky, especially in populated areas where civilian air traffic is dense. Making the call to shoot down an object over the U.S. is an incredibly complex task that never actually occurred until recently.

Tower 22

But even in war zones where the US policy is clear and permissive, it isn't foolproof. Stefan Becket and Kaia Hubbard reported on a January incident at Tower 22 on the border with Syria in 3 American service members killed and dozens injured in drone attack on base in Jordan, U.S. says:

Three American service members were killed and dozens more were injured in an unmanned aerial drone attack on a base in Jordan on Sunday, President Biden and the U.S. military said.
...
Mr. Biden said the attack happened at a base in northeast Jordan, a U.S. ally, close to the border with Syria. A U.S. official said the attack occurred at an outpost known as Tower 22, where roughly 350 U.S. Army and Air Force personnel are deployed, according to the Department of Defense.

U.S. Central Command, which oversees forces in the Middle East, initially put the number of injured at 25, but two U.S. officials soon said that figure had risen to more than 30. CENTCOM confirmed later Sunday night that at least 34 had been injured. Eight of the wounded service members had to be evacuated — some were in critical condition but all were stable, a defense official told CBS News.

How was the drone able to penetrate US defenses and kill or injure more than 10% of the base occupants? Drone that killed US soldiers in Jordan followed American drone onto base, causing confusion by Natasha Bertrand, Oren Liebermann and Haley Britzky explains:

The drone that killed three US Army soldiers and wounded dozens more in Jordan on Sunday approached the US military outpost, Tower 22, around the same time an American drone was returning to the base, which led to uncertainty over whether it was hostile and caused a delay to the US response, two US officials told CNN.

The enemy drone followed the American drone as it approached, but it is not clear whether the enemy drone intentionally followed the American one or if it was a coincidence, one of the officials said.

These problems don't go away if the counter-drone technology is electronic. Confusing the drone's navigation with GPS spoofing or detaching an FPV drone from its pilot using a jammer may prevent it reaching its target but it doesn't render it harmless to innocent bystanders.

The Threat

One notable feature of the war in Ukraine is Russia's sustained campaign against Ukraine's electric grid, which has caused major difficulties especially in the winter. On November 26^th Volodymyr Zelenskyy tweeted:

Last night, Russia attacked Ukraine, launching a record number of strike drones—188—against our people. I want to thank our defenders of the sky for repelling the attack. Around 80 drones were shot down, and more than 90 were lost due to location disruption.

Unfortunately, there has been damage to our critical infrastructure, and the situation in Ternopil remains challenging. All services and repair crews are on-site, doing everything to assist people and restore electricity as quickly as possible.

A mysterious feature has been Ukraine's reluctance to attack Russia's electric grid, especially since their economy and military are completely dependent upon freight trains hauled by electric locomotives. This may have been part of the US' "escalation management".

If whoever is behind the drone incursions in the West decides to switch from reconnaissance to strike, they are likely to follow the Russian example and strike the grid by attacking the transformers. US transformers have already been shot and damaged by unknown attackers, presumably right-wing militia. FPV drones would cause far more than the $15M damage in that attack. Large grid transformers are expensive, long lead-time items, completely defenseless against drone attack. A coordinated attack on a number of them would have a major, long-lasting impact on the US grid, similar to a major coronal mass ejection.

The Future

The cost of precision strike, both at short and longer ranges, has been reduced by at least an order of magnitude and likely much more. Further reductions are very likely. Typically, when the cost of something is greatly reduced, you get a lot more of it. What is likely to be the effect of greatly increasing the supply of precision strike?

The impact of precision strike technology on the warfare of non-state armed groups: case studies on Daesh and the Houthis by Max Mutschler, Marius Bales and Esther Meininghaus examines one effect that has already happened:

Precision strikes from a distance are a common practice of state warfare. However, the global proliferation of precision strike technologies, like missiles and armed drones, makes such weapons progressively available to non-state armed groups (NSAGs). We look at Daesh in Syria and Iraq and at the Houthis in Yemen as two case studies to analyse the consequences of this proliferation for non-state warfare.

The Houthis, with help from Iran, have used precision strike systems much more sophisticated than the ones I deal with here to cause a major disruption to world trade, and to cause Western nations to expend vastly more expensive systems in a futile attempt to stop them.

The war in Ukraine spawned hundreds of new drone manufacturers. This incredibly competitive market has driven costs down to a point where groups with far less funding than the drug cartels, and even individuals, can acquire precision strike capabilities. In the US civilian use of drones is currently regulated by the FAA. Drones are required to be registered and the areas where they are permitted to fly are restricted. They are not regarded as weapons, but what does the 2^nd Amendment say about drones if they are regarded as weapons?

Many of the same policy issues arise when it comes to individuals defending themselves against drone attacks. Electronic defenses such as jamming and GPS spoofing are illegal. "Kinetic" defenses such as shotguns are legal in some cases, but restricted by the risks to bystanders. Passive defenses such as "cope cages" on vehicles would be illegal under safety regulations, and on houses by planning regulations.

Potential future technological developments may greatly increase the difficulty of defending against drone attacks by making it impossible to interfere with their navigation. Current high-end cruise missiles such as Ukraine's R360-Neptune use inertial navigation, but because it drifts over time it requires regular updates from GPS to maintain accuracy. This is similar to the navigation systems in commercial aircraft. These systems are too large and expensive for drones.

Source

But, as Sandia Labs announced in Revolutionary Quantum Compass Could Soon Make GPS-Free Navigation a Reality dramatic increases in accuracy and reductions in cost and size for inertial navigation are on the way:

Now, scientists are attempting to make a motion sensor so precise it could minimize the nation’s reliance on global positioning satellites. Until recently, such a sensor — a thousand times more sensitive than today’s navigation-grade devices — would have filled a moving truck. But advancements are dramatically shrinking the size and cost of this technology.

For the first time, researchers from Sandia National Laboratories have used silicon photonic microchip components to perform a quantum sensing technique called atom interferometry, an ultra-precise way of measuring acceleration.
...
Besides size, cost has been a major obstacle to deploying quantum navigation devices. Every atom interferometer needs a laser system, and laser systems need modulators.

“Just one full-size single-sideband modulator, a commercially available one, is more than $10,000,” Lee said.

Miniaturizing bulky, expensive components into silicon photonic chips helps drive down these costs.

“We can make hundreds of modulators on a single 8-inch wafer and even more on a 12-inch wafer,” Kodigala said.

Source

Quantum inertial systems for commercial aircraft are undergoing flight tests in the UK and the US. The image shows the unit Boeing is testing. Mentour Pilot's video provides an accessible introduction to the technology.

Even with Sandia's progress in reducing the size and cost of quantum intertial navigation, the technology is currently far out of reach for low-cost drones. But if progress were to continue it would mean that a drone, or more likely a drone swarm, could navigate to any target whose coordinates were known without using any external assistance that could be jammed or spoofed. Electrical grid transformers would be one such target.

Driver Distraction Technology

2024-11-19T08:00:00.004-08:00

Not this hand-off

In the aftermath of the 737 MAX crashes, I wrote First We Change How People Behave.:

The fundamental problem of autonomous vehicles sharing roads is that until you get to Level 5, you have a hand-off problem. The closer you get to Level 5, the worse the hand-off problem.

Three years earlier, Paul Vixie was more specific in Disciplining the Unoccupied Mind:

Simply put, if you give a human brain the option to perform other tasks than the one at hand, it will do so. No law, no amount of training, and no insistence by the manufacturer of an automobile will alter this fact. It's human nature, immalleable. So until and unless Tesla can robustly and credibly promise an autopilot that will imagine every threat a human could imagine, and can use the same level of caution as the best human driver would use, then the world will be better off without this feature.

Follow me below the fold for an update on the hand-off problem.

The 737 MAX crashes were a specific, especially difficult, case of the hand-off problem. It wasn't that the automation recognized a situation it couldn't cope with and initiated a hand-off to the humans. It was that the automation was wrong about the situation and the pilots had to decide to override it:

In testing performed in a simulator, Boeing test pilots recreated the conditions aboard Lion Air Flight 610 when it went down in the Java Sea in October, killing 189 people. The tests showed that the crew of the 737 MAX 8 would have only had 40 seconds to respond to the Maneuvering Characteristics Augmentation System’s (MCAS’s) attempts to correct a stall that wasn’t happening before the aircraft went into an unrecoverable dive, according to a report by The New York Times.

While the test pilots were able to correct the issue with the flip of three switches, their training on the systems far exceeded that of the Lion Air crew—and that of the similarly doomed Ethiopian Airlines Flight 302, which crashed earlier this month. The Lion Air crew was heard on cockpit voice recorders checking flight manuals in an attempt to diagnose what was going on moments before they died.

Christine Negroni's What people don’t get about why planes crash stresses the hand-off problem:

"In the crash of an Asiana Airlines Boeing 777 landing in San Francisco in 2013, investigators determined that a contributing factor was the pilots’ over-reliance on automated systems which led to an erosion in their flying skills. The investigation of the fatal flight of an Air France Airbus A330 from Rio de Janeiro to Paris in 2009 led to the conclusion that the complexity of the fly-by-wire airplane befuddled the pilots.

The 737 Max probes suggest another variation on the conundrum: Technology intended to protect against pilot error trapped the pilots. Helpless in the cockpit, they were unable to do as Captain Sully did and save the day."

Source

Now, Road and Track's Joe Kucinski reports that Tesla Has the Highest Fatal Accident Rate of All Auto Brands, Study Finds:

The study was conducted on model year 2018–2022 vehicles, and focused on crashes between 2017 and 2022 that resulted in occupant fatalities. Tesla vehicles have a fatal crash rate of 5.6 per billion miles driven, according to the study; Kia is second with a rate of 5.5, and Buick rounds out the top three with a 4.8 rate. The average fatal crash rate for all cars in the United States is 2.8 per billion vehicle miles driven.

How is this possible when "Autopilot" is standard on Teslas? The analyst behind the study explains:

So, why are Teslas — and many other ostensibly safe cars on the list — involved in so many fatal crashes? “The models on this list likely reflect a combination of driver behavior and driving conditions, leading to increased crashes and fatalities,” iSeeCars executive analyst Karl Brauer said in the report. “A focused, alert driver, traveling at a legal or prudent speed, without being under the influence of drugs or alcohol, is the most likely to arrive safely regardless of the vehicle they’re driving.”

Precisely. It seems very likely that, lulled by the experience of being cocooned in high-tech safety systems, the drivers were not focused or alert, ready to take over from the automation in a split second when danger threatened.

Nvidia vs. Intel

2024-11-08T08:00:00.004-08:00

NV1-based Diamond Edge
Swaaye, CC-By-SA 3.0

Today Nvidia replaced Intel in the Dow Jones Industrial Average with a market cap of about $3.6T, about the same as Apple, as against Intel's market cap about 33 times less.

That is a long way from Curtis Priem's kitchen table, a $2.5M A-round from Sutter Hill and Sequoia, and the NV1.

1.5C Here We Come

2024-10-29T08:00:00.072-07:00

Source

John Timmer's With four more years like 2023, carbon emissions will blow past 1.5° limit is based on the United Nations' Environmental Programme's report Emissions Gap Report 2024. The "emissions gap" is:

the difference between where we're heading and where we'd need to be to achieve the goals set out in the Paris Agreement. It makes for some pretty grim reading. Given last year's greenhouse gas emissions, we can afford fewer than four similar years before we would exceed the total emissions compatible with limiting the planet's warming to 1.5° C above pre-industrial conditions.
...
The report ascribes this situation to two distinct emissions gaps: between the goals of the Paris Agreement and what countries have pledged to do and between their pledges and the policies they've actually put in place.

Source

Back in 2021 in my TTI/Vanguard talk I examined one of these gaps, the one between the crypto-bros' energy consumption:

The leading source for estimating Bitcoin's electricity consumption is the Cambridge Bitcoin Energy Consumption Index, whose current central estimate is 117TWh/year.

Adjusting Christian Stoll et al's 2018 estimate of Bitcoin's carbon footprint to the current CBECI estimate gives a range of about 50.4 to 125.7 MtCO2/yr for Bitcoin's opex emissions, or between Portugal and Myanmar.

and their rhetoric:

Cryptocurrencies assume that society is committed to this waste of energy and hardware forever. Their response is frantic greenwashing, such as claiming that because Bitcoin mining allows an obsolete, uncompetitive coal-burning plant near St. Louis to continue burning coal it is somehow good for the environment.

But, they argue, mining can use renewable energy. First, at present it doesn't. For example, Luxxfolio implemented their commitment to 100% renewable energy by buying 15 megawatts of coal-fired power from the Navajo Nation!.

Second, even if it were true that cryptocurrencies ran on renewable power, the idea that it is OK for speculation to waste vast amounts of renewable power assumes that doing so doesn't compete with more socially valuable uses for renewables, or indeed for power in general.

Source

Note that the current CBECI estimate shows that Bitcoin's energy consumption has increased 43% since 2021, a 12.7%/yr increase.

Follow me below the fold for more details of the frantic greenwashing, not just from the crypto-bros but from the giants of the tech industry that aims to ensure that:

Following existing policies out to the turn of the century would leave us facing over 3° C of warming.

Luxxfolio wasn't an exception. The latest example of Bitcoin greenwashing comes from Hunterbrook Media:

TeraWulf Inc. (NASDAQ: $WULF) brands itself as a “zero-carbon Bitcoin miner” — and claims its commitment to renewable energy will help it land AI data center contracts. But the New York Power Authority, which supplies 45% of the facility’s energy, told Hunterbrook Media: “None of the power that NYPA provides the firm can be claimed as renewable power.”

The rest of TeraWulf’s power is sourced from the New York grid, which is less than half zero-carbon, according to the New York Independent System Operator, the organization responsible for managing the state’s wholesale electric marketplace.

The only way TeraWulf can legally substantiate its zero-carbon claims is by purchasing renewable energy credits (RECs), according to New York and federal regulators, but a TeraWulf spokesperson confirmed that the company has not done so. “Without the REC, there is no legal claim to the renewable attributes of electricity,” a spokesperson for the New York State Energy Research and Development Authority confirmed in an email to Hunterbrook.

These lies were just the start, Hunterbrook documents lies about most aspects of their business. Note TeraWulf's pivot to AI. In Bitcoin Miners Take Divergent Paths Six Months After Revenue ‘Halving’, David Pan explains that TeraWulf is part of a trend:

Six months after rewards for validating transactions on the Bitcoin network were reduced by half, crypto mining companies are choosing between two divergent paths to remain viable.

Public miners including MARA Holdings, Riot Platforms and CleanSpark are keeping the Bitcoin they produce with the expectation that the digital asset will rise in value. At the same time, an increasing number of companies are spending more on developing data centers that power artificial intelligence applications.

It isn't just the crypto-bros who are apperently lying about using renewables. Back in July Adele Peters revealed that Amazon says it hit a goal of 100% clean power. Employees say it’s more like 22%:

Today, Amazon announced that it hit its 100% renewable electricity goal seven years early. But a group of Amazon employees argues that the company’s math is misleading.

A report from the group, Amazon Employees for Climate Justice, argues that only 22% of the company’s data centers in the U.S. actually run on clean power. The employees looked at where each data center was located and the mix of power on the regional grids—how much was coming from coal, gas, or oil versus solar or wind.

Amazon, like many other companies, buys renewable energy credits (RECs) for a certain amount of clean power that’s produced by a solar plant or wind farm. In theory, RECs are supposed to push new renewable energy to get built. In reality, that doesn’t always happen. The employee research found that 68% of Amazon’s RECs are unbundled, meaning that they didn’t fund new renewable infrastructure, but gave credit for renewables that already existed or were already going to be built.

And in August Amy Castor and David Gerard posted How to fix AI’s ghastly power consumption? Fake the numbers!:

Big tech uses a stupendous amount of power, so it generates a stupendous amount of CO2. The numbers are not looking so great, especially with the ever-increasing power use of AI.

So the large techs want to fiddle how the numbers are calculated!

Companies already have a vast gap between “market-calculated” CO2 and actual real-world CO2 production. The scam works a lot like carbon credits. Companies cancel out power used on the coal/gas-heavy grid in northern Virginia by buying renewable energy credits for solar energy in Nevada.

So in 2023, Facebook listed just 273 tonnes of “net” CO2 and claimed it had hit “net zero” — but it actually generated 3.9 million tonnes.

In practice, RECs don’t drive new clean energy or any drop in emissions — they only exist for greenwashing.

It gets worse. Large techs are already the largest buyers of RECs. So they’re lobbying the Greenhouse Gas Protocol organization to let them report even more ludicrously unrealistic numbers.

RECs currently have to be on the same continent at the same time of day. Amazon and Facebook propose a completely free system with no geographical constraints. They could offset coal power in Virginia with wind power from Norway or India.

This will make RECs work even more like the carbon credit market — where companies can claim hypothetical “avoided” CO2 against actual, real-world CO2.

Source

In Data center emissions probably 662% higher than big tech claims. Can it keep up the ruse? Isabel O'Brien reinforced the message:

Amazon is the largest emitter of the big five tech companies by a mile – the emissions of the second-largest emitter, Apple, were less than half of Amazon’s in 2022. However, Amazon has been kept out of the calculation above because its differing business model makes it difficult to isolate data center-specific emissions figures for the company.

As energy demands for these data centers grow, many are worried that carbon emissions will, too. The International Energy Agency stated that data centers already accounted for 1% to 1.5% of global electricity consumption in 2022 – and that was before the AI boom began with ChatGPT’s launch at the end of that year.

AI is far more energy-intensive on data centers than typical cloud-based applications. According to Goldman Sachs, a ChatGPT query needs nearly 10 times as much electricity to process as a Google search, and data center power demand will grow 160% by 2030. Goldman competitor Morgan Stanley’s research has made similar findings, projecting data center emissions globally to accumulate to 2.5bn metric tons of CO2 equivalent by 2030.

In the meantime, all five tech companies have claimed carbon neutrality, though Google dropped the label last year as it stepped up its carbon accounting standards. Amazon is the most recent company to do so, claiming in July that it met its goal seven years early, and that it had implemented a gross emissions cut of 3%.

Because the tech giants are funnelling vast amounts of cash to Nvidia for hardware to train AIs to, for example, tell people to eat at Angus Steakhouse, or put glue on pizza, convince them that black people's IQ is inferior to whites, hallucinate patient's responses to doctors, persuade teens to commit suicide, and so on they will need lots of power. The smart miners have figured out that their access to lots of power is worth more to the AI bubble than the Bitcoin it could mine. Especially since the halvening. The market has figured this out too:

while the shares of the majority of the companies have underperformed Bitcoin’s more than 60% rally this year with future mining revenue constrained, traders appear to be voting which strategy will succeed, with those embracing AI posing the largest gains.

MARA and Riot, two of the largest publicly traded Bitcoin miners and both “hodlers,” have seen their shares slump 20% and 36%, respectively, this year.

On the other hand:

Northern Data AG is examining a possible sale of its crypto mining business to free up funds for expanding its artificial-intelligence operations.

The Frankfurt-listed company, whose main shareholder is stablecoin issuer Tether Holdings Ltd., would use proceeds from the sale of Peak Mining to focus on its AI solutions unit, it said in a statement Monday. Shares of Northern Data jumped as much as 12% on the news, and were up 9.8% as of 12:06 p.m. in Frankfurt.

The big tech companies are desperate for power:

They are continuing to burn coal at plants that were due to shut down in Montana, Omaha (Google & Facebook), Utah, Georgia and Wisconsin:

“This is very quickly becoming an issue of, don’t get left behind locking down the power you need, and you can figure out the climate issues later,” said Aaron Zubaty, CEO of California-based Eolian, a major developer of clean energy projects. “Ability to find power right now will determine the winners and losers in the AI arms race. It has left us with a map bleeding with places where the retirement of fossil plants are being delayed.”
Morgan Stanley estimates that:

The datacenter industry is set to emit 2.5 billion tonnes of greenhouse gas (GHG) emissions worldwide between now and the end of the decade, three times more than if generative AI had not been developed.
S&P Global Commodity Insights:

noted that only 54 gigawatts of the US coal industry is projected to be powered off by 2030 – down 40 percent from a prediction made in July last year. The total number of coal plants retired by 2050 is still expected to be roughly the same, but the pace of retirement from now to the end of the decade will be significantly slower compared to last year's estimates.
...
Coal plants can credit their new lease on life to the datacenter industry, which is expanding and upgrading existing bit barns as well as building new facilities. The age of AI requires lots of energy – Google search powered by AI alone is expected to use ten times the power of a more traditional information request, according to the International Energy Agency's (IEA) January report.
Microsoft signed a 20-year contract to restart Three Mile Island:

Constellation Energy shut down the Unit 1 reactor in 2019 — not the one that melted down in 1979, the other one — because it wasn’t economical. Inflation Reduction Act tax breaks made it viable again, so Constellation went looking for a customer. Microsoft has signed up for 835 megawatts for the next 20 years.
...
Other mothballed nuclear reactors want to restart for data centers, including Palisades in Michigan and Duane Arnold in Iowa. These both shut down because renewables and natural gas were cheaper — but the data centers need feeding.

TMI Unit 1 should be back online in 2028, going into the strained local grid — so when the AI bubble pops, the clean-ish power will still be there.
Google and Amazon have signed deals for Small Modular Reactors (SMRs), and so has Oracle, but:

Google has signed a deal with California startup Kairos Power for six or seven small modular reactors. The first is due in 2030 and the rest by 2035, for a total of 500 megawatts.

Amazon has also done three deals to fund SMR development.
...
Only three experimental SMRs exist in the entire world — in Russia, China, and Japan. The Russian and Chinese reactors claim to be in “commercial operation” — though with their intermittent and occasional hours and disconcertingly low load factors, they certainly look experimental.

Like general AI, SMRs are a technology that exists in the fabulous future. SMR advocates will talk all day about the potential of SMRs and gloss over the issues — particularly that SMRs are not yet economically viable.

Kairos doesn’t have an SMR. They have permission to start a non-powered tech demo site in 2027. Will they have an approved and economically viable design by 2030?

Of course, the nuclear options won't add CO2 to the atmosphere, but they won't come on line until after we've breached 1.5C. The result is the rapidly increasing "emssions gap" of the large tech companies. But the problem is even worse than it appears. In my EE380 talk I discussed the carbon emmissions from Bitcoin's hardware:

Bitcoin's growing e-waste problem by Alex de Vries and Christian Stoll concludes that:

Bitcoin's annual e-waste generation adds up to 30.7 metric kilotons as of May 2021. This level is comparable to the small IT equipment waste produced by a country such as the Netherlands.
That's an average of one whole MacBook Air of e-waste per "economically meaningful" transaction.

Source

Why does Bitcoin generate so much e-waste?:

The reason for this extraordinary waste is that the profitability of mining depends on the energy consumed per hash, and the rapid development of mining ASICs means that they rapidly become uncompetitive. de Vries and Stoll estimate that the average service life is less than 16 months. This mountain of e-waste contains embedded carbon emissions from its manufacture, transport and disposal. These graphs show that for Facebook and Google data centers, capex emissions are at least as great as the opex emissions.

Lindsay Clark's GenAI's dirty secret: It's set to create a mountainous increase in e-waste points out that AI has the same problem:

Computational boffins' research claims GenAI is set to create nearly 1,000 times more e-waste than exists currently by 2030, unless the tech industry employs mitigating strategies.

The study, which looks at the rate AI servers are being introduced to datacenters, claims that a realistic scenario indicates potential for rapid growth of e-waste from 2.6 kilotons each year in 2023 to between 400 kilotons and 2.5 million tons each year in 2030, when no waste reduction measures are considered.

Assuming that the tech giants eventually succeed in generating profits from their massive investments in AI data centers, it is likely that the economic life of Nvidia's hardware is longer than that of Bitmain's mining rigs. But the investment is much bigger, so it is likely that the capex emissions from AI data centers add greatly to the overall climate impact of AI. Even if they never make profits, the capex emissions from the current build-out will still be in the atmosphere.

Interestingly, the mainstream media has started to pay attention. Back in June the Washington Post's Evan Halper and Caroline O'Donovan's AI is exhausting the power grid. Tech firms are seeking a miracle solution reported on the latest shiny object:

So near the river’s banks in central Washington, Microsoft is betting on an effort to generate power from atomic fusion — the collision of atoms that powers the sun — a breakthrough that has eluded scientists for the past century. Physicists predict it will elude Microsoft, too.

The tech giant and its partners say they expect to harness fusion by 2028, an audacious claim that bolsters their promises to transition to green energy but distracts from current reality.

Even if they could "harness fusion by 2028", it would be too late to avoid 1.5C. But no-one has yet built a fusion reactor with a positive power output, so the 2028 claim is obvious BS. Pay attention to their actions not words:

In fact, the voracious electricity consumption of artificial intelligence is driving an expansion of fossil fuel use — including delaying the retirement of some coal-fired plants.
...
The data-center-driven resurgence in fossil fuel power contrasts starkly with the sustainability commitments of tech giants Microsoft, Google, Amazon and Meta, all of which say they will erase their emissions entirely as soon as 2030. The companies are the most prominent players in a constellation of more than 2,700 data centers nationwide, many of them run by more obscure firms that rent out computing power to the tech giants.

“They are starting to think like cement and chemical plants. The ones who have approached us are agnostic as to where the power is coming from,” said Ganesh Sakshi, chief financial officer of Mountain V Oil & Gas, which provides natural gas to industrial customers in Eastern states.

And this month the New York Times' David Gelles' The A.I. Power Grab reported that Nvidia was also pushing the "AI will solve the climate" fantasy:

Nvidia’s chips are incredibly power-hungry. As the company rolls out new products, analysts have taken to measuring the amount of electricity needed to power them in terms of cities, or even countries.

There are already more than 5,000 data centers in the U.S., and the industry is expected to grow nearly 10 percent annually. Goldman Sachs estimates that A.I. will drive a 160 percent increase in data center power demand by 2030.

Dion Harris, Nvidia’s head of data center product marketing, acknowledged that A.I. was creating a huge spike in power usage. But he said that over time, that demand would be offset as A.I. made other industries more efficient.

“There is sort of a myopic view on the data center,” he said, “but not really an understanding that a lot of those technologies are going to be the main way that we’re going to innovate our way to a net-zero future.”

Apart from continuing to burn fossil fuels as fast as they can and signing deals that won't make a difference until after the world has committed to 1.5C, what are the tech giants doing? Just like the crypto-bros, they are greenwashing, and spinning ludicrous futures to prevent current action. Here, for example, is Eric Schmidt:

Eric Schmidt, the former chief executive of Google, recently said that the artificial intelligence boom was too powerful, and had too much potential, to let concerns about climate change get in the way.

Schmidt, somewhat fatalistically, said that “we’re not going to hit the climate goals anyway,” and argued that rather than focus on reducing emissions, “I’d rather bet on A.I. solving the problem.”

Schmidt at Sun

Full disclosure: I reported to Schmidt at Sun Microsystems, and my opinion of him is less negative than most of my then peer engineers. But I would not expect him to sacrifice immediate profits for the health of the planet. He is right that “we’re not going to hit the climate goals anyway", but that is partly his fault. Even assuming that he's right and AI is capable of magically "solving the problem", the magic solution won't be in place until long after 2027, which is when at the current rate we will pass 1.5C. And everything that the tech giants are doing right now is moving the 1.5C date closer.

It Was Ten Years Ago Today

2024-10-07T08:00:00.006-07:00

Ten years ago today I posted Economies of Scale in Peer-to-Peer Networks . My fundamental insight was:

The income to a participant in a P2P network of this kind should be linear in their contribution of resources to the network.

The costs a participant incurs by contributing resources to the network will be less than linear in their resource contribution, because of the economies of scale.

Thus the proportional profit margin a participant obtains will increase with increasing resource contribution.

Thus the effects described in Brian Arthur's Increasing Returns and Path Dependence in the Economy will apply, and the network will be dominated by a few, perhaps just one, large participant.

In the name of blatant self-promotion, below the fold I look at how this insight has held up since.

Experience in the decade since has shown that this insight was correct.

Source

The insight applies to Proof Of Work networks; for the entire decade Bitcoin mining has always been dominated by five or fewer mining pools. As I write this AntPool, ViaBTC and F2Pool have had more than 50% of the hashrate over the last week. Even within those pools, the vast expense of mining rigs, the data centers to put them in, and the power to feed them make economies of scale essential.

Source

The insight applies to Proof Of Stake networks at two levels:

Block production: over the last month almost half of all blocks have been produced by beaverbuild.
Staking: Yueqi Yang noted that:

Coinbase Global Inc. is already the second-largest validator ... controlling about 14% of staked Ether. The top provider, Lido, controls 31.7% of the staked tokens,
That is 45.7% of the total staked controlled by the top two.

Source

In addition all these networks lack software diversity. For example, as I write the top two Ethereum consensus clients have nearly 70% market share, and the top two execution clients have 82% market share.

Economies of scale and network effects mean that liquidity in cryptocurrencies is also highly concentrated. In Decentralized Systems Aren't I wrote:

There have been many attempts to create alternatives to Bitcoin, but of the current total "market cap" of around $2.5T Bitcoin and Ethereum represent $1.75T or 70%. The top 10 "decentralized" coins represent $1.92T, or 77%, so you can see that the coin market is dominated by just two coins. Adding in the top 5 coins that don't even claim to be decentralized gets you to 87% of the total "market cap".

The fact that the coins ranked 3, 6 and 7 by "market cap" don't even claim to be decentralized shows that decentralization is irrelevant to cryptocurrency users. Numbers 3 and 7 are stablecoins with a combined "market cap" of $134B. The largest stablecoin that claims to be decentralized is DAI, ranked at 24 with a "market cap" of $5B.

Protocol	Revenue	Market
	$M	Share %
Lido	304	55.2
Uniswap V3	55	10.0
Maker DAO	48	8.7
AAVE V3	24	4.4
Top 4		78.2
Venus	18	3.3
GMX	14	2.5
Rari Fuse	14	2.5
Rocket Pool	14	2.5
Pancake Swap AMM V3	13	2.4
Compound V2	13	2.4
Morpho Aave V2	10	1.8
Goldfinch	9	1.6
Aura Finance	8	1.5
Yearn Finance	7	1.3
Stargate	5	0.9
Total	551

Similar effects apply to "Decentralized Finance". In DeFi Is Becoming Less Competitive a Year After FTX’s Collapse Battered Crypto Muyao Shen wrote:

Based on the [Herfindahl-Hirschman Index], the most competition exists between decentralized finance exchanges, with the top four venues holding about 54% of total market share. Other categories including decentralized derivatives exchanges, DeFi lenders, and liquid staking, are much less competitive. For example, the top four liquid staking projects hold about 90% of total market share in that category,

Based on data on 180 days of revenue of DeFi projects from Shen's article, I compiled this table, showing that the top project, Lido, had 55% of the revenue, the top two had 2/3, and the top four projects had 78%.

Because these systems, if successful, cannot be decentralized, the cryptosphere doesn't care about the fact that they aren't. In Deconstructing ‘Decentralization’: Exploring the Core Claim of Crypto Systems Prof. Angela Walch explains what the label "decentralized" is actually used for:

the common meaning of ‘decentralized’ as applied to blockchain systems functions as a veil that covers over and prevents many from seeing the actions of key actors within the system. Hence, Hinman’s (and others’) inability to see the small groups of people who wield concentrated power in operating the blockchain protocol. In essence, if it’s decentralized, well, no particular people are doing things of consequence.

Going further, if one believes that no particular people are doing things of consequence, and power is diffuse, then there is effectively no human agency within the system to hold accountable for anything.

In other words, it is a means for the system's insiders to evade responsibility for their actions.

Warning: Slow Blogging Ahead

2024-10-03T08:00:00.000-07:00

Vicky & I have recently acquired two major joint writing assignments with effective deadlines in the next couple of months. And I am still on the hook for a Wikipedia page about the late Dewayne Hendricks. This is all likely to reduce the flow of posts on this blog for a while, for which I apologize.

Dewayne Hendricks RIP

2024-09-23T14:30:00.000-07:00

Source

Dewayne Hendricks, my friend of nearly four decades, passed away last Friday at age 74. His mentors were Buckminster Fuller and Paul Baran. He was a pioneer of wireless Internet connectivity, a serial entrepreneur, curator of an influential e-mail list, and for the last 30 years on the organizing committee of the Asilomar Microcomputer Workshop.

For someone of his remarkable achievements he has left very little impression on the Web. An example is his Linkedin profile. Below the fold I collect the pieces of his story that I know or have been able to find from his other friends. If I can find more I will update this post. Please feel free to add information in the comments.

Wayne State University

Dewayne was a student at Wayne State, where he got into systems programming for the IBM 370. They ran the Michigan Terminal System on a 512K 370/155. He tried to run the University of Newcastle's CMTS, an experimental version of MTS that didn't use dynamic address translation, and ran into performance problems. He worked with Larry Chace at the University of Illinois to get it running on their 1+2M 360/75. After Chace rewrote it to swap pages to their 2301 drum storage it ran well.

Southern Illinois University

While at Southern Illinois Univerity in the '70s he worked for Buckminster Fuller and continued his involvement in IBM systems programming, now for VM/370. Melinda Varian, historian of that era of IBM's groundbreaking SHARE user group, wrote in VM and the VM Community: Past, Present, and Future:

Dewayne Hendricks reported at SHARE XLII, in March, 1974, that he had successfully implemented MVT-CP handshaking for page faulting, so that when MVT running under VM took a page fault, CP would allow MVT to dispatch another task while CP brought in the page. At the following SHARE, Dewayne did a presentation on further modifications, including support for SIOF and a memory-mapped job queue. With these changes, his system would allow multi-tasking guests actually to multi-task when running in a virtual machine. Significantly, his modifications were available on the Waterloo Tape.

Dewayne became the chairman of the Operating Systems Committee of the SHARE VM Project. Under his guidance, the Committee prepared several detailed requirements for improvements to allow guest systems to perform better. At SHARE XLV, in 1975, the Committee presented IBM with a White Paper entitled Operating Systems Under VM/370, which discussed the performance problems of guests under VM and the solutions that customers had found for these problems. Many of the solutions that Dewayne and others had found, such as PAGEX, made their way into VM fairly quickly, apparently as the result of customers’ persistence in documenting them. By SHARE 49, Dewayne was able to state that, “It is now generally understood that either MFT or MVT can run under VM/370 with relative batch throughput greater than 1.” That is to say, they had both been made to run significantly faster under VM than on the bare hardware. Dewayne and others did similar work to improve the performance of DOS under VM.

Amateur Radio

Dewayne was a major figure in developing and sustaining the use of amateur packet radio:

He has been involved with radio since receiving his amateur radio operator's license as a teen. He currently holds official positions in several national non-profit amateur radio organizations and is a director of the Wireless Communications Alliance, an industry group representing manufacturers in the unlicensed radio industry.

In particular:

Back in 1986, he ported the popular KA9Q Internet Protocol package to the Macintosh, allowing the Macintosh platform to be used in packet radio networks. Today, thousands of amateur radio operators worldwide use the NET/Mac system he developed to participate in the global packet radio Internet. This system continues to be developed and deployed by the amateur radio service.

Dewayne was a member of the Amateur Radio Digital Communications Grants Evaluation Team from 2021 to his death. ARDC grants around $5M/year:

ARDC makes grants to projects and organizations that are experimenting with new ways to advance both amateur radio and digital communication science. Experimentation by amateur radio operators has benefited society in many ways, including the development of the mobile phone and wireless internet technology. ARDC envisions a world where all such technology is available through open source hardware and software, and where anyone has the ability to innovate upon it. To see examples of the types of grants we make, go to https://www.ardc.net/grants/.

Tetherless Access

One of the many ahead-of-their-time companies Dewayne started was Tetherless Access. He co-founded it with Charlie Brown in 1990 to develop wireless Metropolitan Access Networks. It went public in 1996 on NASDAQ and folded two years later. The idea was to use the 900MHz unlicensed spectrum to distribute Internet connectivity from a base station via point-to-point links, in contrast to Metricom's Ricochet service, started by Dewayne's mentor Paul Baran, which four years later used mesh network technology in the same spectrum.

Source

Tetherless Access launched a testbed network which:

Started in Fall ‘96

Covered 35 mi area in south bay

Delivered from ISDN to 30 Mbps bandwidth

Used both licensed and unlicensed equipment (Part 15 and 97)

NSF Projects

Dewayne was involved in a number of NSF funded experiments in using wireless to connect remote communities:

Prior to forming Dandin Group, he was the General Manager of the Wireless Business Unit for Com21, Inc. He joined Com21 following an opportunity to participate as the Co-Principal Investigator in the National Science Foundation’s Wireless Field Tests for Education project. The project sucessfully connected remote educational institutions to the Internet. The test sites ranged from rural primary schools in Colorado, USA to a University in Ulaan Bataar, Mongolia.

Com21 was founded by Dewayne's mentor Paul Baran.

Ulan Bator rooftop
Courtesy Glenn Tenny

The PI for connecting Mongolia to the Internet for the first time in 1996 was the "Cursor Cowboy" Colonel Dave Hughes, an equally remarkable character who was a pioneer of bulletin boards starting in 1981!. The NSF funded a 256Kbit/s satellite dish, the State Department shipped it to the US embassy in Ulan Bator via the "diplomatic pouch", and Dewayne and Glenn Tenney travelled via Beijing to deploy 900MHz links across the city.

The Dandin Group, Dewayne's next company was:

a partner in the Advanced Networking Project with Minority Serving Institutions (AN-MSI) an EDUCAUSE project funded by the National Science Foundation. The project's purpose is to provide improved communication services, including Internet access, to underserved minority and tribal-nation institutions. Because these institutions are frequently in remote locations which currently lack communication infrastructures, Internet-linked services delivered by wireless networks offer the most appropriate and cost-effective approach to connecting their communities to the world and to each other.

The project description is here. NSF Awards $6 Million to Help Minority Schools Prepare for Advanced Computer Networks is EDUCAUSE's press release:

National Science Foundation (NSF) Director Rita Colwell announced last week at EDUCAUSE '99 that the foundation has awarded almost $6 million over four years to help institutions of higher learning that traditionally serve minority communities prepare for the next generation of information technology and computer networks. The grant will be administered by EDUCAUSE.

Developing Countries

Dewayne was not just active in getting Internet service to under-served communities with the NSF. The bio on his website states:

Tetherless Access was one of the first companies to develop and deploy Part 15 unlicensed wireless metropolitan area data networks using the TCP/IP protocols. He has participated in the installation of these networks in other parts of the world including: Kenya, Tonga, Mexico, Canada and Mongolia.

Source

Amara Angelica reported that Tonga first to go wireless for telecommunications:

"We’re replacing the entire existing telecom infrastructure with a wireless IP [Internet protocol] network," says Dewayne Hendricks, CEO of Fremont-based Dandin Group and former general manager of Com21’s wireless business unit. "Since the country is a monarchy, there was only one guy to convince, Crown Prince Tupouto’a, and then we just went for it."

Hendricks’ firm plans to replace Tonga Telecom’s aging landline system—which still uses mechanical relays—with a broadband wireless network for data, video and telephony (using voice over IP). It will run at 30Mbps with user access at 2Mbps and 10Mbps by the end of next year. "We can get all the spectrum we want," Hendricks says.

The prince’s objective, Hendricks says, is to convert the country’s largely agricultural workforce, which has an astonishing 95 percent literacy rate, into knowledge workers, such as programmers. The government launched the Royal School of Science for Distance Learning last year, using Internet connections to allow students to take courses at international universities. There are just fewer than 100,000 people in Tonga scattered across 170 islands.

"We’re going to an Internet-style mesh network," says Hendricks. MMDS, which some carriers are using to deliver broadband services, won’t scale well for an IP network, he says. Hendricks, a technical advisor to the FCC on ultrawideband (UWB) technology, is considering UWB for the network.

Tonga had about 11,000 households and 6,500 phone customers, with an 8-year wait to get a phone. The goals of the project were to deliver 30Mbit/s IP to each home for a customer end budget of $450.

FCC Technological Advisory Council

Dewayne was one of the inaugural members of the Federal Communications Commission's Technological Advisory Council, launched on April 30^th 1999, together with luminaries such as Vint Cerf, AT&T CTO David Nagel, CERFnet founder Susan Estrada and many others. He remained a member through the fourth TAC formed in 2005.

Wired Article

In the January 2002 edition of Wired, Brent Hurtig's Digital Cowboy focused on Dewayne's work on the reservation:

At Turtle Mountain Chippewa Reservation in North Dakota, he's installing a wireless network. In its initial form, the system will meet FCC requirements governing frequency, power, and transmission technology. But not for long. Hendricks' mission is to build the best system possible - even if it's illegal - and he intends to use every tool at his disposal. Should the FCC crack down, the tribal leaders will hoist the flag of Native American sovereignty, asserting that they can do whatever they want with the sky above their reservation.

Dewayne's work on the reservation, in Tonga and elsewhere was an attempt to demonstrate the problems with the obsolete US spectrum allocation policy:

There's no sensible reason why Americans shouldn't have inexpensive, ubiquitous, high-performance broadband access, Hendricks says. Using technologies that are already available or in fast-track development, everyone could enjoy reliable, fully symmetrical wireless at T1 speed or better. No more digital divide. No more last-mile problem. No more compromises. The only things standing in the way are the FCC, Congress, and "other people who just don't get it."

EE380 Talks

Dewayne gave three talks to Stanford's EE380 symposium. The first one was apparently "in the ‘90s on wireless MANs" of which I have so far found no record.

The second was on 3^rd May 2000 entitled Wiring Tonga: From the Ground Up and the Sky Down. The abstract was:

One of the biggest barriers today standing in the way of deployment of advanced wireless communications systems turns out not to be the technology, but restrictions related to regulatory policies. This presentation will discuss the nature of these barriers and how they have affected the development of wireless data systems over the years.

The speaker will also discuss on-going work in which he is involved to use advanced wireless technology to deploy multiservice IP systems as part of infrastructure-development projects in the Kingdom of Tonga and with Native American groups in the US, and how such projects are able to deal with the limitations imposed by conventional regulatory barriers.

The slides are here.

The third was on 5^th March 2014 and entitled Inventing a New Internet: Learning from Icarus. The abstract read:

From a future historical perspective, are we descendants of Icarus? Is our Internet like Icarus' wings? Are our protocols, ciphers and codes, brilliant capabilities built on immature engineering, which like Icarus' wax and feathers, are capable of taking us to great heights, but systematically flawed? For a brief historical moment, humanity has flown high like Icarus, on a vulnerable first generation Internet platform. Which as been used for securing and using distributed ideas, arts, media science, commerce, and machines. Promising brilliant futures with the arrival of networked things, autonomous personalized services and immersive media. But, now our first generation Internet , built on a fragile global network of vulnerable codes and protocols, is falling apart, like Icarus' wings, through a triple shock from:

Massive dotcom data stalker economy built on mining of terabytes personal data.

Ubiquitous criminal penetration of financial and identity networks, on our devices, in the cloud.

Pervasive state intruders at all levels and every encrypted hardware and software node.

Humans eventually conquered the barriers to flight and learned to build durable and resilient aircraft. Similarly, humans must learn to build a more reliable, private and secure Internet for communications, innovation and commerce. We will share our thoughts on how we might go about the design of a more durable and resilient Internet:

How prepared is the Internet for future human benefit?

What are the attributes of a future more durable internet?

What are the existing assets that could be harnessed?

What needs to be developed?

Dewayne's slides for this talk are here. Video of the talk is on YouTube

dewayne-net

For many years Dewayne with impeccable taste curated dewayne-net, an e-mail list to which he sent links, most he found but some contributed by his friends. A typical e-mail would have the title of the linked post, a link, and enough of the content to encourage recipients to read the whole thing. The last e-mails were two on 19^th August, as it happens both links that I had sent him earlier. I have been one of the more frequent contributors, although only perhaps 20% of my contributions passed the curatorial filter. Prof. Dave Farber's IP list is a similar and I believe even longer-standing list; he and Dewayne exchanged links fairly often.

As an example of the list in full flow, lets take April 2022. That month he sent 66 e-mails, many about the COVID pandemic and the war in Ukraine, obviously both top of mind at the time. But they included topics including satellite tracking of commercial aircraft, the Kessler syndrome, the problems of the US patent system, cybercrime, microplastics, banned math textbooks in Florida, and Elon Musk's purchase of Twitter. I am already greatly missing this window into Dewayne's eclectic set of interests.

Dewayne on YouTube

Towards Open Spectrum was a keynote to ABC2 Conference at Cisco's EMEA HQ in January 2004.
Dewayne Hendricks on The Internet of Things: Extended Interview 8^th January, 2009.
Inventing a New Internet: Learning from Icarus 5^th March 2014.
Dewayne Hendricks (WA8DZP) on creating abundant bandwidth 2012.
Second Curve Internet 31^st March, 2015.

Lie Down WIth Dogs, Get Up WIth Fleas

2024-09-17T08:00:00.000-07:00

Source

It is generally quite difficult to upset the denizens of a wretched hive of scum and villainy by further besmirching their reputation, but recently the Trump family has succeeded.

Below the fold I explain how they did it, and why the denizens of the wretched hive are not happy.

Matt Levine identified the fundamental mechanism at play here:

But you don’t have to just take your counterparties as you find them; you can work to encourage and develop bad ones. This is often a matter of product development: If you build a product that does nothing for sophisticated professionals but that is really good for noisy addicted gamblers, you will attract exactly the right sort of counterparty. This arguably explains much of crypto.

The additional point he omitted is that, in order to "attract exactly the right sort of counterparty" you have to advertise the product. This will bring it to the attention of others seeking the same kind of counterparty. They will seek to leverage your product design for their own ends.

Trump has perfected the art of attracting "the right sort of counterparty". From Trump University to the banks he stiffed in his multiple bankruptcies, to the stockholders of Truth Social his counterparties have always been the "sucker at the poker table". Key to this success has always been moving on to the next scam as the previous one implodes.

By the side of the road to ~~Damascus~~ the White House the former crypto-skeptic President encountered a small crowd of large cryptocurrency companies, including Coinbase, Ripple, Andreessen Horowitz and Jump Capital. They were waving immense sums of money. Trump underwent an instant conversion to the cryptocurrency faith, announcing he would make the US the "Bitcoin capital of the world".

Apart from the general idea that "more is better", why would these immense sums of money be attractive to this alleged billionaire? Apart from the costs of running a Presidential campaign, some of which ends up in his own pocket, there is the need to pay his legions of lawyers, and the looming judgements that these lawyers failed to avert, including $88.3M before interest to E. Jean Carroll and $454M before interest in his New York fraud case.

With Truth Social's Q2 financials reporting a loss of 1,911% of revenue, which was down 30% year-on-year, and with the stock down 80% from its peak, it was clearly time to move on.

On 31^st August Jasper Goodman posted Crypto is the new Trump family business. Ethics watchdogs have concerns. rating a well-deserved "Well Duh!":

Trump’s eldest sons are gearing up to launch a new cryptocurrency venture called World Liberty Financial, which is already receiving a big social media boost from their father. Government ethics watchdogs say the project could create a conflict of interest if Trump returns to the White House next year.

Trump has vowed to enact an array of pro-crypto policies in a bid to win votes — and campaign cash — from digital asset enthusiasts in recent months. Now, he’s weaving the overtures into his pitch for his sons’ forthcoming startup.

While the details of Eric Trump and Donald Trump Jr.'s crypto endeavor haven’t yet been announced, it could stand to benefit from a second Trump administration’s approach to support the industry.
...
CREW identified more than 3,700 conflicts of interest during Trump’s first term.

But critics said the crypto project differs because it could align his family’s financial interest with policy changes that he would be poised to enact.

“The prior conflicts and illegalities took advantage of preexisting loopholes,” said Norman Eisen, an ethics lawyer who served in the Obama White House and later helped build the first impeachment case against Trump. “Here, Trump appears to be promising to create the loopholes while his family is simultaneously designing a business venture to exploit them.”

The headline on World Financial's "coming soon" web page is "World Liberty Financial - The Only Crypto DeFi Platform supported by Donald J. Trump". There are dozens of "DeFi Platforms", but the unique selling point of World Liberty is that it is "supported by Donald J. Trump". So the counterparties it is intended to attract are the gullible members of the Trump cult, the ones who have allegedly made him a billionaire.

With this unique selling point the actual product doesn't need to be very good to compete with the market leader, Uniswap, for its chosen counterparties. As Molly White reported, it wasn't:

CoinDesk has gotten hold of a white paper for this supposed World Liberty Financial, which they note appears to be a clone of Dough Finance, a crypto lending platform that was hacked for around $2 million just a month and a half ago. The hack was not a sophisticated one, and instead exploited sloppiness on behalf of the development team.

Of course, many people interested in gullible counterparties noticed all the hype, and they didn't have to wait to hack the actual product launch to get in on the action:

The project hasn’t even launched yet, and it’s off to a bumpy start. First, Donald Trump Jr. had to issue a statement to try to stop people from buying up all the fake tokens purporting to be associated with their murky project. Then, Twitter accounts for Lara and Tiffany Trump were both compromised and used to send tweets announcing a supposed token launch. “This is a scam!!!” tweeted Eric, himself retweeting the tweet from his wife’s account containing the scam token address.

Nic Carter tweeted:

can't secure twitter accounts but i'm sure they'll be able to protect an (already hacked) defi protocol from hacks
— nic carter (@nic__carter) September 4, 2024

The reason for this fiasco was that, as usual, the Trumps hired "only the best people". Sam Kessler, Danny Nelson & Cheyenne Ligon dug into the project's white paper in Inside the Trump Crypto Project Linked to a $2M DeFi Hack and Former Pick-Up Artist:

The document and other reporting describe a borrowing and lending service strikingly similar to Dough Finance, a recently hacked blockchain app built by four people listed as World Liberty Financial team members. Other participants include all three of Trump's sons (including 18-year-old Barron, who is identified as the project's "DeFi visionary"), financiers and e-commerce influencers.

The leader of the Dough Finance team was Chase Herro. When it came time to move on, he obviously found in the Trumps the ignorant and greedy counterparty he needed. Zeke Faux and Muyao Shen also piled on with Behind the Trump Crypto Project Is a Self-Described ‘Dirtbag of the Internet’:

Herro, a fast-talking 39-year-old who shows off his fancy cars and private-jet rides on social media, is an unknown in the crypto world. More than a dozen prominent digital-asset investors said in interviews they had never heard of him. The only crypto project with which he was publicly affiliated attracted only a few million dollars and suffered a devastating hack. A token he promoted on influencer Logan Paul’s podcast dropped 96% afterward. In one speech in 2018, he called himself “the dirtbag of the internet” and said that regulators should “kick s---heads like me out.”

“You can literally sell s--- in a can, wrapped in piss, covered in human skin, for a billion dollars if the story's right, because people will buy it,” Herro said about crypto in a 2018 YouTube video recorded as he drove in a Rolls-Royce. “I'm not going to question the right and wrong of all that.”

Source

Jasper Goodman reported on the cryptosphere's reaction to this incompetent blatant grift in ‘A huge mistake’: Trump’s crypto allies cringe over family’s startup:

“This is a huge mistake,” said Nic Carter, a Trump supporter who is a founding partner at the crypto-focused venture capital firm Castle Island Ventures. “It looks like Trump’s inner circle is just cashing in on his recent embrace of crypto in a kind of naive way, and frankly it looks like they’re burning a lot of the good will that’s been built with the industry so far.”
...
“It’s a very typical playbook of smaller operators or more amateur operations in the crypto space to try to generate a lot of hype before revealing the details,” said Austin Campbell, an adjunct professor at NYU’s Stern School of Business who previously led the risk and portfolio management operation at the crypto firm Paxos. “That makes them susceptible to all sorts of nonsense.”
...
One crypto industry representative in Washington, granted anonymity because of sensitivities around criticizing Trump, described having “a laundry list of concerns.” A big worry is that that it could reflect poorly on the industry as it pushes for policy changes that would help legitimize the sector.

“Maybe it doesn’t move the needle for most people, but if this thing is hacked or regular folks lose money on it or it opens up the door for the SEC to investigate the team, it only looks like it has downside risk,” Carter said. “It looks to have very little upside risk.”

As with Trump University and Truth Social, the whole point is for "regular folks" to lose money. But, if you're a candidate positioning yourself as pro-crypto in the hope of getting large checks from the cryptosphere, making this obvious to everyone but the cult members is a bad idea.

PS: Wikipedia explains the title of this post here.

"Owning" e-books

2024-09-03T08:00:00.000-07:00

The basic aspiration of the LOCKSS Program when we started a quarter century ago was to enable libraries to continue their historical mission of collecting, preserving, and providing readers with access to academic journals. In the paper world libraries which subscribed to a journal owned a copy; in the digital world they could only rent access to the publisher's copy. This allowed the oligoply academic publishers to increase their rent extraction from research and education budgets.

LOCKSS provided a cheap way for libraries to collect, preserve and provide access to their own copy of journals. The competing e-journal preservation systems accepted the idea of rental; they provided an alternate place from which access could be rented if it were denied by the publisher.

Similarly, libraries that purchased a paper book owned a copy that they could loan to readers. The transition to e-books meant that they were only able to rent access to the publisher's copy, and over time the terms of this rental grew more and more onerous.

Below the fold I look into a recent effort to mitigate this problem.

Gennie Gebhart's 2019 Publishers Should be Making E-Book Licensing Better, Not Worse reports on another turn of the e-book rental screw:

When Macmillan releases a new book, library systems will be able to purchase only one digital copy for the first eight weeks after it’s published. Macmillan is offering this initial copy for half-price ($30), but that has not taken away the sting for librarians who will need to answer to frustrated users. In large library systems in particular, readers are likely to experience even longer hold queues for new Macmillan e-book releases. For example, under the new Macmillan embargo, the 27 branches of the San Francisco Public Library system, serving a city of nearly 900,000 people, will have to share one single copy right when the demand for the new title is the greatest.
...
After the two-month embargo period ends, libraries will be welcome to purchase additional copies of the e-book under normal terms, which aren’t great to begin with: typically, a $60 price tag for an e-book that can only be lent out to one user at a time for two years or 52 lends, whichever comes first. After that, the library has to license another e-book.

Gebhart's use of "purchase" here is misleading, they are signing a two-year lease. Macmillian can make life miserable for libraries and their readers because they are "too big to care". Gebhart explains:

In a July memo, CEO John Sargent says the publisher’s move is motivated by “growing fears that library lending was cannibalizing sales” of new e-books and a need to “protect the value of your books during their first format publication,” but fails to present any evidence to back up his claims. (He also ignores existing, consistent evidence to the contrary.)

On 13^the August, the Independent Publishers Group and the Digital Public Library of America announced a deal:

Through this landmark collaboration between IPG and DPLA, libraries around the country will now have the power to purchase and own in perpetuity, rather than merely license, tens of thousands of ebook and audiobook titles from dozens of independent publishers. The agreement will empower libraries to fulfill their mission to provide access to books for readers nationwide. Publishers such as Austin Macauley, Arcadia Publishing, Dynamite Entertainment, Dover Publications and JMS Books, alongside dozens of other renowned indie publishers, are participating in the deal.

Of course, none of the major publishers would agree to this. The press release explains:

Now, libraries will be able to purchase books through the Palace Marketplace, a non-profit ebook and audiobook platform developed by The Palace Project in consultation with libraries, for libraries. The Palace Project provides libraries with tools and resources that allows patrons to access the broadest possible range of ebooks and audiobooks sourced from the widest variety of sources. Among the tens of thousands of books that are now available for libraries to own and lend out digitally are contemporary award-winning titles, like Leticia Aguilar’s spellbinding memoir Leaving Patriarchy Behind, Michael Nicholson’s The Mosaic Escalator, a mind-bending adventure story nominated for the Best Fiction Book Award by the Golden Book Awards 2024, as well as classics like Elizabeth Bowen’s debut novel The Hotel.
...
“After more than 12 years explaining to policy makers and publishers why existing license models for ebooks prevent libraries from fulfilling their mandates, this agreement is a win that libraries, publishers and authors should celebrate together,” said Christina de Castell, Chief Librarian & CEO of the Vancouver Public Library. “Now, libraries will be able to take steps to share and preserve authors’ ebooks as we have always shared and preserved their print books. I hope this agreement is a model that leads other publishers towards more flexible licensing terms, so libraries can continue to be the home for our collective knowledge in our increasingly digital world.”

Details of how the technology works are scarce, although the Palace code is on github. The DPLA's Everyone should have access to knowledge claims:

Palace Marketplace, formerly DPLA Exchange, is an e-content acquisitions platform that works seamlessly with the Palace app. With Palace Marketplace, libraries can customize diverse collections beyond the bestsellers, including non-English titles. Member libraries benefit from DPLA’s ongoing publisher negotiations for new, more favorable licensing terms, and have access to our collection of more than 11,000 open access ebooks.

So it appears that "members" have access to books they "purchased" hosted at DPLA. Once again, as with e-journal preservation systems such as Portico, this is a welcome development but it isn't a solution to the underlying problem. It is in fact another instance of the same problem, because the member libraries are renting access to DPLA's copy. As I understand it, these libraries would be able to download a copy, and preserve it themselves outside the Palace ecosystem, but it isn't clear whether the terms of use would allow them to provide their readers access to the preserved copy.

The key barrier to implementing true "purchase" rather than "rental" is the First-sale Doctrine:

The first-sale doctrine (also sometimes referred to as the "right of first sale" or the "first sale rule") is an American legal concept that limits the rights of an intellectual property owner to control resale of products embodying its intellectual property. The doctrine enables the distribution chain of copyrighted products, library lending, giving, video rentals and secondary markets for copyrighted works (for example, enabling individuals to sell their legally purchased books or CDs to others).

But it doesn't apply to digital copies:

The first-sale doctrine does not neatly fit transfers of copies of digital works because an actual transfer does not actually happen—instead, the recipient receives a new copy of the work while, at the same time, the sender has the original copy (unless that copy is deleted, either automatically or manually). ...

E-books have the same issue. Because the first sale doctrine does not apply to electronic books, libraries cannot freely lend e-books indefinitely after purchase. Instead, electronic book publishers came up with business models to sell the subscriptions to the license of the text.

Mary Minow's 2017 Future of Libraries – Need First Sale for ebooks explains the importance of first sale for libraries:

It is essential to libraries, and the term existential would not be too great a term to use, to be able to own digital files, and care for them via preservation and library lends (e.g. to one person at a time) just as they do with print. Can readers count on books being available a year or two or five after publication? The existence of libraries has made this possible from their inception until now.

The DPLA echoes this:

“Copyright’s ‘first sale doctrine’ is important to readers and libraries for a number of reasons, not least of which is the way that it facilitates long-term preservation and access,” noted Lyrasis CEO John Wilkin. “Licensing models have threatened this critical underpinning, which is needed for preservation of the cultural record. Lyrasis is excited to collaborate with DPLA and IPG in the creation of a model that will make preservation and access of digital content possible.”

Alas, even if the first-sale barrier were removed, the experience of the LOCKSS program shows that the problem would remain. In the paper world building a local collection was essential for libraries to provide service to their current readers. The costs and effort involved in doing so were inescapable. The fact that doing so safeguarded access for future readers was a no-cost, no-effort side-effect. But in the digital world building a local collection is irrelevant to providing service to current readers. They get access from the publisher. The costs and effort devoted to building a local collection are only in the interest of future readers. Thus, given strained budgets, these costs must be minimized. The choice between maintaining a local collection, and simply outsourcing the task of supporting future readers to a service by signing a small check is a no-brainer.

Worse, three other considerations weigh in the balance:

The publishers are highly motivated to ensure that current readers access their copy, because this provides them data they can sell to advertisers.
The publishers view many local collections as a bigger risk of their content leaking than a small number of concentrated archives, which they expect will be better secured.
Economies of scale mean a centralized archive is likely to have a lower cost per unit content than a distributed one.

The LOCKSS Program devoted a great deal of engineering effort to reducing the capital and operational cost, and increasing the security of, two generations of an appliance that libraries could use to build a local collection of the e-journals to which they subscribed. But in the end most libraries that cared about protecting access for future readers to this content chose to sign checks to centralized archives. Of course, it didn't help that the publishers of the most expensive content would not allow libraries to use the appliance to collect it.

2024 Optical Media Durability Update

2024-08-27T08:00:00.000-07:00

Six years ago I posted Optical Media Durability and discovered:

Surprisingly, I'm getting good data from CD-Rs more than 14 years old, and from DVD-Rs nearly 12 years old. Your mileage may vary.

Here are the subsequent annual updates:

It is time once again for the mind-numbing process of feeding 45 disks through the readers to verify their checksums, and yet again this year every single MD5 was successfully verified. Below the fold, the details.

Month	Media	Good	Bad	Vendor
01/04	CD-R	5x	0	GQ
05/04	CD-R	5x	0	Memorex
02/06	CD-R	5x	0	GQ
11/06	DVD-R	5x	0	GQ
12/06	DVD-R	1x	0	GQ
01/07	DVD-R	4x	0	GQ
04/07	DVD-R	3x	0	GQ
05/07	DVD-R	2x	0	GQ
07/11	DVD-R	4x	0	Verbatim
08/11	DVD-R	1x	0	Verbatim
05/12	DVD+R	2x	0	Verbatim
06/12	DVD+R	3x	0	Verbatim
04/13	DVD+R	2x	0	Optimum
05/13	DVD+R	3x	0	Optimum

The fields in the table are as follows:

Month: The date marked on the media in Sharpie, and verified via the on-disk metadata.
Media: The type of media.
Good: The number of media with this type and date for which all MD5 checksums were correctly verified.
Bad: The number of media with this type and date for which any file failed MD5 verification.
Vendor: the vendor name on the media

The drives I use from ASUS and LG report significant numbers of read errors from the CDs but verify the MD5s correctly. I didn't notice them reporting any read errors from the DVDs. An off-brand drive fails to read the CDs, but read one of the older DVDs with no read errors.

Surprisingly, with no special storage precautions, generic low-cost media, and consumer drives, I'm getting good data from CD-Rs more than 20 years old, and from DVD-Rs nearly 18 years old. Your mileage may vary. Tune in again next year for another episode.

I also checked my Memorex NetBSD1.2 CD written in October 1996. It has checksums generated by cksum(1), all of which verified correctly despite a number of read errors. So that CD-R is delivering good data after nearly 28 years.

Astroturfing

2024-08-20T08:00:00.001-07:00

I seem to be stuck on the theme of cryptocurrency gaslighting with two weeks ago More Cryptocurrency Gaslighting and one week ago Greenwashing. Now I look at cryptocurrency gaslighting in the political arena, where it is termed astroturfing:

it is defined as the process of seeking electoral victory or legislative relief for grievances by helping political actors find and mobilize a sympathetic public, and is designed to create the image of public consensus where there is none. Astroturfing is the use of fake grassroots efforts that primarily focus on influencing public opinion and typically are funded by corporations and political entities to form opinions.

Donald Trump, 2019

Currently, the crypto-bros have poured money into primaries, defeated several incumbents deemed to be insufficiently crypto-friendly, and have accumulated an immense war-chest for November's general election. This pot of gold was enough to turn Trump from crypto-skeptic to telling Maria Bartiromo:

Who knows, maybe we’ll pay off our $35 trillion dollar [national debt], hand them a little crypto check, right? We’ll hand them a little Bitcoin and wipe away our $35 trillion

Below the fold I discuss the gaslighting the cryptosphere is using in their massive attempt to purchase "regulatory clarity", and what the scale of this investment suggests about the profits they expect to garner if it succeeds.

Why does astroturfing exist? Lobbyists going to politicians and saying "I know your constituents don't want you to vote X, but if you do I'll give you a boiat-load of money" make the politicians uncomfortable. The lobbyists need to cover this unpleasant reality with a fig-leaf. So on 10^th May Crypto Attitudes in Swing States, a poll by Harris for the Digital Currency Group appeared. It polled 1201 registered voters in April. Jasper Goodman's Crypto is Trump’s new weapon against Biden reported that:

A poll released by crypto industry groups this week showed that more than 20 percent of voters in six key swing states identify crypto as a major issue. A separate nationwide survey of registered voters commissioned by the crypto firm Paradigm found that crypto ownership is higher among communities of color and young people — key constituencies that helped boost Biden in 2020 but are now proving to be challenging for him to win over.

Crypto has gained support from politicians on the right who tout it as an alternative to the mainstream financial system. The Paradigm poll found earlier this year that crypto owners favor Trump over Biden, 48 percent to 39 percent, with 13 percent undecided.

Source

Three weeks later Molly White had had time to analyze the poll and in Cryptocurrency companies have raised over $135 million to influence US elections this cycle, and they’re just getting started she wrote:

In reality, their poll of 1,201 American voters in six swing states is a masterclass in spin. 69% of the people surveyed had “very” or “somewhat” negative opinions towards crypto, but DCG is crowing that “Among a sizeable contingent of voters, crypto attracts a level of interest that translates into pro-crypto sentiments across the board” and advising political candidates to adopt pro-crypto stances to attract voters. Many of the questions were framed ambiguously — such as the “Crypto is a major issue I’m considering during the next election” one — which allowed DCG to put “21% agree” in big letters while obfuscating the fact that those who agreed were not necessarily crypto supporters. In fact, if you do the math, more than 100 of the 252 people (~40%) who said they felt crypto was a major issue in upcoming elections do not feel positively towards crypto — in other words, there's a contingent of fellow skeptics among that 21%, who are likely hoping for candidates who might support more aggressive approaches against the industry.

White extracted the agree/disagree data from the Harris poll and generated a useful chart. From it I extracted the questions that two out of three respondents somewhat or strongly disagreed with in this table.

Disagreement	Question
80%	Crypto is a major issue I'm considering in the next election
74%	Crypto is more equitable than the traditional financial system
73%	Crypto is meant for people like me
73%	I pay attention to political candidates positions on crypto
71%	It is easier for people to be financially successful with crypto ...
65%	Crypto is the future of transacting
65%	I'm concerned about legislation that would interfere with crypto

This is not a pro-crypto sample. I have to agree with White that DGC is gaslighting the results. She concluded:

Frankly, it just doesn't seem likely to me that a substantial number of people are single-issue crypto voters and would be swayed to vote for a candidate they might not otherwise support because of it. Many of the people celebrating Trump's recent crypto U-turn were probably already planning to vote for him, and I doubt there were many crypto-skeptical people out there previously who didn't otherwise support Trump, but who intended to vote for him because of his former position on the industry.

The most worrying answer is that 73% of the sample are not paying attention to candidates' positions on cryptocurrencies; this gives the candidates license to ignore their constituents' anti-crypto views.

Source

Covered by this bogus fig-leaf, the tsunami of money started flowing. May 30^th saw some of the first reports of money with Hannah Miller and Olga Kharif's Andreessen Horowitz Donates $25 Million to Fairshake PAC and Molly White's Cryptocurrency companies have raised over $135 million to influence US elections this cycle, and they’re just getting started. White went on to start Follow the crypto, a site devoted to tracking the cryptosphere's political fundraising and donations. Much of this post is based on White's research.

Initially, the crypto-bros had consideable success by pouring money into defeating crypto-skeptics in primaries, as the Emily Wilkins' Crypto industry super PAC is 33-2 in primaries, with $100 million for House, Senate races reported on 26^th June:

A super PAC bankrolled by a small group of crypto companies has backed the winning candidate in 33 of the 35 House and Senate primary races it entered.

Fairshake PAC kicks off the general election season with a strong track record and at least $100 million to spend on crypto-friendly candidates for Congress.

Wilkins made two important observations:

The PAC has spent tens of millions already on ads that rarely mention crypto. Instead, the messages they deliver are about “fairness” and “integrity.”

Note that the crypto-bros aren't fooled by their gaslighting; they know that the vast majority of the voters are anti-crypto. Molly White's Pointing its arsenal at our friends makes the point:

I’ve noted before that, despite the industry’s attempts to convince incumbent politicians, candidates, campaign managers, and anyone else who will listen that there is a major bloc of single-issue voters who will vote for people with pro-crypto policy stances, the people in charge of actually running strategy don’t seem to truly believe it. Ads run by the Fairshake PAC and related crypto-focused super PACs make no mention of cryptocurrency, for example, suggesting that those behind them understand that it’s not an issue that’s likely to sway voters.

Wilkins' second observation is:

Of the $160 million in total contributions Fairshake has raised since it was founded, around $155 million — or 94% — can be traced back to just four companies: Ripple, Andreesen Horowitz, Coinbase and Jump Crypto.

And note the classic astroturfing; a PAC funded by a few huge companies claiming to represent the "40% of American adults now own crypto". This is more gaslighting. Other crypto-friendly sites claim "14% of US adults own cryptocurrencies".

Grifters like Trump know which side their bread is buttered. Already by 7^th June Alexandra Ulmer was writing Trump pitches himself as 'crypto president' at San Francisco tech fundraiser:

Republican presidential candidate Donald Trump presented himself as a champion for cryptocurrency and slammed Democrats' attempts to regulate the sector during a San Francisco fundraiser on Thursday, three sources present told Reuters.

Trump raised $12 million from the fundraiser hosted by tech venture capitalists David Sacks and Chamath Palihapitiya at Sacks' home in the swanky Pacific Heights neighborhood.

Four days later David Pan reported Trump Meets Bitcoin Miners in His Latest Pro-Crypto Overture:

Several Bitcoin miners met with former president Donald Trump at Mar-a-Lago on Tuesday night, according to Matthew Schultz, executive chairman at crypto mining company CleanSpark Inc.

Trump told attendees that he loves and understands cryptocurrency, adding that Bitcoin miners help to stabilize energy supply from the grid, according to Schultz. Trump said he’d be an advocate for miners in the White House, Schultz added.

This culminated on 27^th July at the recent Bitcoin2024 gathering of the faithful with a bizarre 48-minute rambling monologue that was mostly a rehash of his stump speech.

Six days earlier, Joe Biden had dropped out of the race, disrupting the crypto-bros plans. Molly White observes:

But it does seem that some of the more gullible executives and figureheads within the crypto world have themselves been bamboozled by the questionable numbers and creatively interpreted polls. The feeling among that group is not that they need to pursue a positive relationship with a possible future Harris administration, but rather that this supposed crypto vote will decide the election, and that they are its gatekeepers.

Like me, Cory Doctorow leverages White's research in The largest campaign finance violation in US history:

But the real whale that's backstopping the crypto campaign spending is Coinbase, through its Fairshake crypto PAC. Coinbase has donated $45,500,000 to Fairshake, which is a lot: https://www.coinbase.com/blog/how-to-get-regulatory-clarity-for-crypto But $45.5m isn't merely a large campaign contribution: it appears that $25m of that is the largest illegal campaign contribution by a federal contractor in history, "by far," a fact that was sleuthed out by Molly White: https://www.citationneeded.news/coinbase-campaign-finance-violation/ At issue is the fact that Coinbase is bidding to be a US federal contractor: specifically, they want to manage the crypto wallets that US federal cops keep seizing from crime kingpins. Once Coinbase threw its hat into the federal contracting ring, it disqualified itself from donating to politicians or funding PACs:

How wonderful is it that this great country enjoys the blessings of democracy? Just imagine how much money the companies donating to these PACs stand to make from obtaining "regulatory clarity" if they are "investing" close to $200M in a chance to get it.

Update 23^rd August

I posted this too soon. Enough has happened in the last few days to justify an update:

Will a Harris administration provide "regulatory clarity"? I commented on Hadriana Lowenkron's Harris Supports Policies to Expand Crypto Industry, Aide Says:

"Vice President Kamala Harris will back measures to help grow digital assets, a policy adviser to her campaign said, highlighting efforts to court an emerging cryptocurrency industry expanding its political influence.
But Protos had a skeptical take in Kamala Harris ‘pro-crypto policy’ is yet more fake news:

Another day, another fake Kamala Harris headline. Already this week, we’ve heard all about her non-appointment of Gary Gensler and her non-existent unrealized crypto gains tax, and today we have to point out that the 2024 White House hopeful hasn’t adopted a ‘pro-crypto policy.’
...
Well, rather than Harris suddenly adopting a pro-crypto platform to boost her presidential campaign, Bloomberg published an article.

Specifically, behind its paywall, it ran the quote, “She’s going to support policies that ensure that emerging technologies and that sort of industry can continue to grow.”

That quote related only broadly to the ‘crypto community’ and didn’t come from Harris or her spokesperson. It was actually said by Brian Nelson, a mere campaign advisor.
Exercising traditional bothsiderism, Bloomberg's Editorial Board weighed in with Harris and Trump Shouldn't Pander to the Crypto Crowd:

Just 1% of Americans say they used them for a payment or money transfer last year. More often crypto is used to move money outside of government oversight. That’s helpful for criminals, terrorists and anyone under sanctions. But these are hardly the constituencies that a presidential candidate should be soliciting. Nor should policymakers be encouraging people to park their savings in digital wallets instead of stocks, bonds and other assets that support the real economy.

Instead, candidates should promise to work with Congress and regulators to ensure that the rules applied to cryptocurrencies are consistent with existing laws on fraud, money laundering and sanctions enforcement.
What is the chance that this would happen in a Trump administration?
The answer had come the day before the Editorial Board's vain hand-waving. MacKenzie Sigalos' Trump promotes family’s new crypto platform, ‘The Defiant Ones’ made it crystal clear:

Former President Donald Trump on Thursday promoted a soon-to-launch, Trump Organization crypto platform, “The DeFiant Ones” to his 7.5 million followers on Truth Social
...
“For too long, the average American has been squeezed by the big banks and financial elites,” Trump wrote. “It’s time we take a stand — together.”

The post marks the first time the Republican nominee for president has used his personal platform to promote the as yet unactivated digital bank. Within minutes, his son Donald Trump Jr., shared the post with his 12 million X followers.
Trump's floundering Truth Social posted a $16.4M loss on revenue of $837K (down 30% year-on-year). The stock is currently down 75% from the peak and Trump is widely expected to pull the rug out from holders by dumping his stock in the next few weeks. Thus he needs to find a new grift to replace DJT, and this is clearly it. The probability that "The Defiant Ones" will be rug-pulled is close to 100%.

I should expand on the last point in the original post. Corporations making large political contributions are making a similar calculation to VCs investing in startups. The return on investment each time is likely to be negative, but the return on the occasional winner is enough to wipe out the losses and leave a healthy profit. This leaves two questions that are rarely asked:

How much profit will the contributors garner if their wishes are granted?
Where will this profit come from? In other words, if the contributors are the winners, who are the losers?

The history of cryptocurrencies suggests that the profit will be large and the losers will be the retail "investors".

Greenwashing

2024-08-08T08:00:00.000-07:00

Source

You have only to scan Molly White's Web3 is Going Just Great to realize that you will never find a more wretched hive of scum and villainy than the cryptosphere. Everywhere you look you find lies, grift, fraud, and theft. Below the fold I discuss the latest example, in which a coal company marketing itself as "zero-carbon Bitcoin mining" is just the start.

The legal way to make money from the wretched hives of scum and villainy in the markets is selling short. Recently, a startup called Hunterbrook has developed an innovative business model for doing so. Matt Levine explains:

Hunterbrook hit upon a differentiated media business model:

They would start a newsroom (Hunterbrook Media) to publish general news and investigative journalism.

There’s no paywall and no advertising.

But before each investigative piece is published, the newsroom would send it to Hunterbrook’s affiliated hedge fund (Hunterbrook Capital), which could trade on the news.

The hedge fund’s trading profits can — they hope! — pay the journalists’ salaries.

One way to think of this business model is that Hunterbrook is essentially an activist short-selling hedge fund, like Hindenburg Research: It investigates companies, finds problems, shorts the companies, and then noisily publishes its investigation to draw attention to the problems. But whereas Hindenburg is straightforwardly a short-selling firm, Hunterbrook’s news site is at least theoretically independent of its hedge fund, and if it publishes enough good general news then perhaps it can become more widely read and trusted — and have more stock-price impact — than a pure hedge fund.

Hunterbrook's latest investigation is entitled COAL, CRYPTO, AND FALSE BRANDING: INSIDE TERAWULF’S GREENWASHING MACHINE. Here is Terawulf's pitch:

The future is #ZeroCarbon ♻️⚡️ and $WULF is ahead of the curve.#Bitcoin #BitcoinMining #SustainableMining pic.twitter.com/3aq4OipWld
— TeraWulf (@TeraWulfInc) February 28, 2024

Hunterbrook reveals that Terawulf's claim to run on renewable power is false:

TeraWulf Inc. (NASDAQ: $WULF) brands itself as a “zero-carbon Bitcoin miner” — and claims its commitment to renewable energy will help it land AI data center contracts. But the New York Power Authority, which supplies 45% of the facility’s energy, told Hunterbrook Media: “None of the power that NYPA provides the firm can be claimed as renewable power.”

The rest of TeraWulf’s power is sourced from the New York grid, which is less than half zero-carbon, according to the New York Independent System Operator, the organization responsible for managing the state’s wholesale electric marketplace.

The only way TeraWulf can legally substantiate its zero-carbon claims is by purchasing renewable energy credits (RECs), according to New York and federal regulators, but a TeraWulf spokesperson confirmed that the company has not done so. “Without the REC, there is no legal claim to the renewable attributes of electricity,” a spokesperson for the New York State Energy Research and Development Authority confirmed in an email to Hunterbrook.

But lying about renewables is just the tip of the iceberg. Hunterbrook reports that they lied to get permits:

The team behind TeraWulf promised New York State that it was “not targeting bitcoin or any other cyber currency” in a 2019 application for its facilities seen by Hunterbrook, despite seizing on the idea of bitcoin mining as a strategy as early as 2018.

Hunterbrook is skeptical about Terawulf's commitment to zero-carbon mining because:

The company’s management team, including CEO Paul Prager, trace their roots to Beowulf Energy LLC, an operator of fossil fuel plants that previously revived a struggling coal plant to mine bitcoin. The same TeraWulf executives who claim TeraWulf’s mining is zero-carbon likely continue to manage this coal plant.

Prager apparently has a lavish lifestyle:

Prager has been known to split his time between a “five-bedroom Fifth Avenue co-op” in Manhattan and a sprawling 250-acre estate on Maryland’s Eastern Shore, according to a 2021 profile in Washingtonian. This estate, called Maiden Point Farm, is described as “more like a small village with 20 structures,” including a “main house, English-style gardens, a pool, a tennis pavilion, a squash court,” and even a private gas station for his car collection, which includes multiple Ferraris.

How is this funded?:

Hunterbrook’s analysis of the company’s SEC filings shows TeraWulf’s operations are deeply entwined with a constellation of entities under Prager’s control, making it difficult to discern where TeraWulf ends and Prager’s other interests begin.

Beowulf Electricity & Data, a company owned and controlled by Prager, provides TeraWulf with “infrastructure, construction, operations and maintenance and administrative services necessary to build out and operate certain Bitcoin mining centers” — functions that TeraWulf claims are “readily available from independent entities.”

In 2023, TeraWulf paid $20.3 million to Beowulf for these services, representing 38% of TeraWulf’s combined operating and administrative expenses. It also accounted for more than half of the company’s $29.4 million operating loss for the year. The trend continued into 2024, with TeraWulf recognizing $3.5 million in related party expenses in the first quarter alone, amounting to 21% of its operating and administrative expenses for the quarter.

Source

Prager apears to have another way to fund his lifestyle:

In recent years, Prager has received a significant quantity of TeraWulf shares — through both the related party transactions and financial activities. And in an open letter published early in 2023, he shared that he hadn’t “sold a share” as “evidence of my conviction.” But since mid-2023, Prager has significantly reduced his claimed ownership of the company through a series of stock disposals.

His reported ownership has dwindled, with his holdings falling to around 10 million shares compared to more than 30 million shares in 2023.

Executives of Bitcoin miners have a history of looting their companies. Two years ago Paul Butler wrote in The problem with bitcoin miners:

There’s one group of people for whom bitcoin mining is an extremely lucrative business: executives. Last year, one MARA executive earned over $220 million in cash and stock-based compensation, in a year when the company’s total revenue was $150 million. RIOT’s top five executives collectively took home a more modest $90 million in a year with a net loss.

How was this possible?::

This, I think, points to the crux of the problem. Investors have been happy to provide capital to these companies, looking for anything in the public markets that provides some exposure to bitcoin, without paying much attention to what the companies are doing.

I don’t think it ends well.

It certianly looks like everyone except Hunterbrook wasn't "paying much attention to what the companies are doing".

Like many miners in the wake of One Heck Of A Halvening, Terawulf is trying to pivot to AI. Hunterbrook notes that:

TeraWulf has not resisted the hype, with the management team suggesting it could land a similar deal. CFO Fleury specified in an earnings call earlier this year that, by leasing out all of the 300 available megawatts at Lake Mariner, the company could bring in $300 million to $450 million in hosting revenues a year. TeraWulf’s revenue last year was $69 million.

But this won't happen any time soon:

TeraWulf doesn’t yet have AI-ready facilities, unlike competitors who made the switch to HPC data center hosting years ago. But TeraWulf announced it is on track to build a 2 MW experimental AI data center by the third quarter of this year and plans to build another 20 MW data center with a target completion date of the end of 2024.

I have only hit the highlights of Hunterbrook's work, you should go read the whole thing.

More Cryptocurrency Gaslighting

2024-08-01T08:00:00.000-07:00

SEC vs. Consensys

On 30^th June Amy Castor and David Gerard reported that SEC sues Consensys over MetaMask Swaps and Staking:

The SEC is charging Consensys for unauthorized sales of securities through MetaMask Staking and for failure to register as a broker and a dealer while offering crypto trades and staking services through MetaMask Staking and Swaps. The SEC says that Consensys took $250 million in fees as an unregistered broker.

MetaMask is Consensys’ main money maker — a popular browser-based wallet that also lets you stake ETH and buy and sell crypto via decentralized exchanges with “swaps.”

Consensys' defense strategy poses potentially serious problems for the concept of open source, because they are gaslighting about the software that is the basis for the SEC's complaint being open source. Were the court to (a) fall for their gaslighting but (b) agree with the SEC's complaint it could provide a basis for imposing liability on open source developers.

I am afraid that the explanation for why this is so is necessarily rather long but I and others think that it needs to be understood. So stock up with supplies for the journey and follow me below the fold.

Before the House of Representatives Committee on House Financial Services Subcommittee on Digital Assets, Financial Technology, and Inclusion, William C. Hughes, Senior Counsel & Director of Global Regulatory Matters for Consensys testified thus:

Our firm develops and offers the most popular unhosted wallet software in digital assets, the MetaMask wallet. This offering is open source software that can be downloaded for free from the Apple app or Google extension store, and installed on any Google Chrome browser, iPhone, or Apple desktop computer. The wallet is an interface that allows the user to read and write to the blockchain without any intermediary’s help, akin to how a web browser is a consumer’s direct connection to the open web.

Why was Hughes stressing that the MetaMask wallet was open-source? Consensys responded to the suit by claiming that MetaMask was just software that they wrote:

We are confident in our position that the SEC has not been granted authority to regulate software interfaces like MetaMask.

Consensys claims that their software is open source:

Consensys is a leading Ethereum and decentralized protocols software company. We enable developers, enterprises, and people worldwide to build next-generation applications, launch modern financial infrastructure, and access the decentralized web. ... On this trusted, open source foundation, we are building the digital economy of tomorrow.

Specifically including MetaMask:

Build your dapp using the same infrastructure and APIs powering web3's most popular open-source wallet.

It is true that MetaMask's wallet was originally truly open source, under the MIT license. But as of 4 years ago, the license changed to a Consensys-specific license that prevents commercial use if it serves more than 10K users/month. So in this post "open source" means readable, it does not mean the canonical definition of Free Open Source Software.

But notice Consensys' focus on the wallet software. That is not what the SEC is suing them about. The SEC's complaint is about the services MetaMask provides customers who use the wallet. Consensys' strategy is to conflate the wallet and the services, and claim that all the wallet does is interface to the blockchain.

Lets use an analogy to see why this distinction is important. When I use an open-source browser to access a financial institution's website, such as Vanguard's it receives a bunch of HTML and JavaScript. It isn't technically open source because it doesn't carry an open source copyright license, but I can read it. But it doesn't implement any of the services Vanguard offers me, such as trading mutual funds and transferring dollars to or from my bank account. These are implemented as closed-source, the browser under control of Vanguard's HTML and JavaScript is just a wrapper around them.

Consensys seems to claim that because the "smart contracts" implementing these services are just open source software they do not control and in whose operations they are not involved, they are not acting as a broker because they don't take custody of the customer's tokens.

I should pick a nit. I've written before, for example in Blockchain Gaslighting and More Cryptocurrency Gaslighting, about the crypto-bros habit of loudly and repeatedly making provably false statements in the hope that no-one will bother to check them. This post is, strictly speaking, not about gaslighting because the repeated statements are technically true, but they are intended to be interpreted in a broader context in which they are false.

The reason Consensys' statements are technically true is that the wallet functionality is open source, at least in the sense that it can be read. But, a couple of weeks ago, Jon Reiter and the team at Chainargos came out with a three-part series examining how MetaMask's services actually work that reveals the lie underlying Consensys' arguments:

How MetaMask Swap Really Works uses Etherscan and Sim explorer to trace a single MetaMask swap transaction.
How MetaMask Staking Really Works examines four different staking staking contracts and uses the same techniques to follow a transaction from one of them. It also discusses how one part of MetaMask Bridge works.
How MetaMask Bridge Really Works covers the rest of MetaMask Bridge's functionality, again including a sample transaction.

In each case the structure of the transaction is basically the same. For the details you should read the three posts, but to oversimplify:

The customer's wallet invokes a function, for example MetaMask:Swap Router.swap(), on one of MetaMask's open source contracts. The call parameters identify a specific contract, which in the swap case MetaMask calls the "aggregator" and Chainargos calls the "execution strategy":

It is the algorithm that the user told their broker to use to execute this order
These contracts are closed-source, owned and controlled by Consensys. Part of their function is to transfer the fees for executing the transaction to a Consensys wallet.
The open-source function validates inputs, identifies the "adapter" that implements the requested algorithm, and formats a block of parameters with which to invoke it.
The open-source function transfers control of the customer's tokens to a MetaMask contract. In the case of MetaMask Swap this is an open-source contract, in other cases it is an upgradeable or closed-source contract.
That contract invokes the closed-source, MetaMask owned adapter contract with the encoded parameters.
The adapter performs the transaction, returning the proceeds to the customer after subtracting the fee, which it sends to the Consensys wallet.

Note first that the actual algorithm performing the transaction is closed-source, MetaMask's open-source software simply provides the wrapper exposed to the customer. And second that in each case a closed-source MetaMask contract takes control of the customer's tokens in order to perform the requested transaction. In some cases even the open-source contract is upgradeable, meaning that MetaMask can change what it does on a whim. In some cases the contracts have functions such as pause() demonstrating MetaMask's control over their operation. At minimum, MetaMask needs this control to ensure that the fees for the transaction are routed to their wallet.

Chainargos concludes that this workflow looks similar to the way a broker works:

User enters order including an execution strategy.

This order is sent to an order router.

Router looks up the execution adapter for that strategy.

Router sends the tokens intended to be sold, the adapter address, and the incoming order details to a spender contract.

Spender delegates control over the received tokens as required for order execution.

Spender calls a closed-source contract which effects the execution strategy.

Fees are sent from Spender to the “MetaMask Fee Complex”.

Sale proceeds are returned to the user.

And they point out that:

that steps 4, 5, and 6 look an awful lot like the concepts of “transmittal order” and “transmittal of funds” from here.

"Here" is the Financial Crimes Enforcement Network's Definitions of Transmittal of Funds and Funds Transfer.

The parts of MetaMask that provide services for which Consensys gets fees cannot be open source in the sense of readable. If they were, anyone could copy them, modify them to direct the fees to themselves, and deploy the result. This would kill off Consensys' $250M cash cow. Consensys has a belt-and-braces moat; if they did they would run into the 10K users/month limit of the license for the open source parts of the system. Note that the license appears to relate only to the open source parts, it isn't clear under what terms the closed source parts are used.

Source

The last of the three Chainargos posts was published July 15^th, and on July 17^th a brand-new blog appeared whose first post was Examining SEC v. Consensys by Daniel Barabander. He is Deputy General Counsel at the cryptocurrency VC Variant, so not an unbiased observer. Early in the post, in the table at the right, Barabander simply asserts that Consensys' technology is non-custodial, i.e. that MetaMask does not take custody of the customer's tokens.

How can Barabander make this assertion when it is clear that MetaMask's Spender delegates control over the customer's tokens and then invokes MetaMask's closed-source execution strategy? He relies on the fact that the transaction is atomic, it either succeeds completely, or it fails completely. He writes:

Assuming the adapters are not malicious (which would be another issue entirely), their job is to enable a user to perform an atomic swap—in a single transaction take token A and exchange it for token B. This would mean that transactions are "all or nothing"—either the tokens are swapped or they are not, they cannot end up in possession of the Spender.sol contract or Consensys

If the transaction succeeds, some of the tokens involved end up in possession of Consensys, because they get the fee. But that is not the big problem with this claim. It is true that the transaction either succeeds or fails completely, but what the transaction does with the control it is delegated over the customer's tokens is determined by MetaMask's closed-source adapters. The customer hands control to MetaMask's closed-source code and blindly trusts it will do what MetaMask claims. Barabander admits that this is the case when he assumes that "the adapters are not malicious".

Chainargos points out that:

If you can publish adapters, call setAdapter on the router and modify the input “data” blob you can do anything you like.

Even if you only have some of that power you can do pretty much anything you like with the input tokens.

As I understand it, only MetaMask can do these things. Barabander admits:

While there is some admin functionality on the smart contract for a multi-sig that Consensys may control, there is no evidence to suggest the company actively operates the smart contract’s code.

Chainargos posted a response to Barabander entitled In Re: Examining SEC v. Consensys. They note that the contract has a renounceOwnership() function that Consensys has not called, there are pauseSwaps() and unpauseSwaps() for use when the contract gets hacked, and that the admin functionality has to be used to set up the adapter mapping.

Consensys vs. SEC

In an attempt to preempt the SEC's suit, Consensys had already sued the SEC in the Northern District of Texas. In their complaint they state (my emphasis):

MetaMask is simply an interface — like a web browser — that allows digital asset holders to seamlessly interact with the Ethereum network, including all other users and applications participating on the network. MetaMask neither holds customers’ digital assets nor carries out any transaction functions.

It is a little strange to say that MetaMask the wallet doesn't hold customer's digital assets - what is a "wallet" for? But the MetaMask wallet is unhosted, Consensys doesn't hold customer's digital assets in MetaMask wallets. But Consensys' services accessed via the wallet do take control of them using closed-source "smart contracts" that they control.

It is possible that the court will decide that Consensys is acting as a broker, but will accept their assertion that the software they use to do it is open source. This would be a problem, in that it could be a precedent for assigning liability to the developers of open source software.

Matt Levine Explains Cryptocurrency Markets

2024-07-25T08:00:00.000-07:00

Kanav Kariya

Matt Levine's superpower is his ability to describe financial issues in wonderfully simple terms, and in a section of Monday's column entitled Crypto is for fun he is on top form:

In many cases, the essential attribute of a crypto token is liquidity: What you want, often, is a token that trades a lot, because your goal for the token is to trade it a lot. Real-world utility, a sensible business model, acceptance in real transactions, etc., are all less important than just trading if you think of crypto as a toy market for traders to play with. If a token trades a lot at a high price, that in itself justifies the price, because that is all that is asked of a token: It doesn’t need to have a good underlying business or cash flows; it just has to trade a lot at a high price.

Below the fold I discuss the astonishing story behind this explanation of why wash trading is so rife in cryptocurrencies.

Levine is commenting on Leo Schwartz' The rise and fall of a crypto whiz kid: How 25-year-old Kanav Kariya went from president of Jump Crypto to pleading the Fifth. This tells how Jump Crypto got started as a way to evaluate interns, such as Kanav Kariya, as potential hires:

The firm needed to test the mettle of its would-be staffers—whether they could parse the nuances in financial markets and translate them into algorithmic trading models. But it couldn’t give the temporary hires the keys to the kingdom, with its proprietary strategies and billions of dollars in capital.

Crypto offered a solution. The sector had its own tradable assets, exchanges, and quirks, but it was separated enough from Jump’s world of stocks and bonds that it wouldn’t pose a threat.

Levine comments:

It feels like a useful mental model for crypto: Crypto is what you get when you take the smart ambitious interns at traditional financial firms and put them in charge of their own play market. Only for real money.

Jump Crypto made markets, and for many shitcoins they were the dominant market maker, and thus the reason why the shitcoins traded a lot, and thus the source of their value. This was because they made deals with the shitcoin projects:

Token projects will lend market makers a large supply of tokens so they can kickstart trading. Some firms also negotiate a call option, which gives the market makers the right to buy a chunk of the tokens for a steep discount if the project goes well. Selig says that the inverted structure in crypto—where market makers work with token projects, rather than exchanges—makes some sense, given projects’ need to spur trading activity. It also creates dynamics that would never be allowed in TradFi. While crypto market makers still make money off the trading spreads, the massive windfalls often come from those lucrative call options.

If I'm a shitcoin promoter, I can sell Jump Crypto cheap call options for a bunch of my new shitcoin. They will pump the price by wash trading it. We'll, both get rich; me from selling some of my inflated tokens, and Jump Crypto by exercising the options and selling. Who are we selling to? The retail speculators sucked in by the spectacle of a new token with a rapidly rising "price". This is another version of the VC's List And Dump Schemes described by Fais Khan in "You Don't Own Web3": A Coinbase Curse and How VCs Sell Crypto to Retail.

Schwartz writes:

For a firm like Jump, becoming the market maker for a token project meant unlimited upside with no real financial risk. “If you’re at Jump, you decide which one is going to win,” one crypto exchange founder tells Fortune, speaking on the condition of anonymity to discuss industry dynamics.

And Levine comments:

Man, what a crazy time the crypto boom was. It really did teach a generation of young financial traders that they could build perpetual motion machines: You make a token, you trade it, that makes it go up, the value comes from you trading it, you can do no wrong, you get rich, “unlimited upside with no real financial risk.”

Like most financial schemes, this all worked great at first. Two of the shitcoins for which Jump Crypto made markets were Do Kwon's Terra/Luna pair. Schwartz' article starts by recounting a May 2021 crisis:

[Jump Crypto] had become a kind of silent partner for one of the most high-profile projects in crypto, an algorithmic stablecoin called TerraUSD that was meant to maintain a $1 peg through a complex mechanism tied to a related cryptocurrency called Luna—a careful dance that Jump helped coordinate on the backend by fulfilling trades. But despite the bluster of Terra's swaggering founder, Do Kwon, the stablecoin was failing. It had lost its peg.

Jump Crypto and Do Kwon averted the crisis:

Over the next week, Jump secretly bought up huge tranches of TerraUSD to create the appearance of demand and restore the coin's value to $1, according to court documents. Meanwhile, Kwon "vested" Jump, meaning he agreed to deliver 65 million tokens of Luna to Jump at just $0.40, even though the coin would trade, at times, at more than $90 on exchanges.

Jump ultimately made $1 billion from that agreement alone

Terra loses peg

Like most financial schemes it could not last, and the longer it lasted the bigger the crash:

These dubious heroics succeeded only in staving off the inevitable: When TerraUSD lost its peg again a year later, there was nothing Jump could do. By May 2022, the cryptocurrency had grown in popularity, and its failure was catastrophic. Some $40 billion in investors' money evaporated into thin air in a matter of days.

The details of Jump Crypto's involvement with Terra/Luna were revealed because the SEC's case against Terraform Labs and Kwon was based partly on a whistleblower from Jump Crypto. The testimony will probably feature in the criminal case against Kwon, currently awaiting extradition from Montenegro.

Accelerated Computing

2024-07-16T08:00:00.000-07:00

Source

One big problem of the economies of the US and the UK is the cult of the CEO, and the resulting flood of CEO hagiographies that appear after a surge in their company's stock price. These aren't harmless fluff pieces, they contribute to a CEO mindset that is profoundly destructive — see Elon Musk for one example. Will Hutton writes:

But decades of being congratulated and indulged for the relentless pursuit of their own self-interest has turned the heads of too many of our successful rich. They really believe that they are different: that they owe little to the society from which they have sprung and in which they trade, that taxes are for little people. We are lucky to have them, and, if anything, owe them a favour.

Below the fold I continue the "Old man yells at cloud" theme of recent posts by trying to clarify an aspect of the current Jensen Huang hagiography.

Ed Zitron's must-read The Shareholder Supremacy traces the idea of the super-hero CEO revealed by a soaring stock price to:

the famous Dodge vs. Ford Motor Co. case that would define — and ultimately doom — modern capitalism, and in many ways birth the growth-at-all-costs Rot Economy.

The Michigan Supreme Court found that "a business corporation is organized and carried on primarily for the profit of the stockholders [and that] the powers of the directors are to be employed for that end," and intimated that cash surpluses should not be saved to invest in upcoming projects, but distributed to shareholders, because Ford had shown that it was good at making money. Ford was directly forbidden from lowering prices and raising employee salaries, and forced to issue a dividend.

To be clear, the statement around corporations’ duty toward shareholders was made “obiter dicta.” This means it was not actually legally binding, despite over a hundred years of people acting as if it was.

Zitron goes on to detail how "Neutron" Jack Welch destroyed General Electric while being lionized as a super-star CEO. I can testify to his malign influence because, during my time at Sun Microsystems, Scott McNealy was seduced by it.

Now on to the latest CEO whose soaring stock price has caught the attention of the CEO hagiographers, Jensen Huang. Don't get me wrong. I was there when he was a really good startup CEO, and unlike many others he has grown with the company in a very impressive way.

In The Envy of Everyone, M. G. Siegler remarks on:

this incredibly prescient profile of co-founder and CEO Jensen Huang back in 2017 by Andrew Nusca (in naming him Fortune's 2017 'Businessperson of the Year').

At the time Siegler tweeted:

“Video games was our killer app — a flywheel to reach large markets funding huge R&D to solve massive computational problems.”

Genius foresight. Sounds obvious now. Was not then.

Siegler was commenting on Andrew Nusca's 2017 profile entitled This Man Is Leading an AI Revolution in Silicon Valley—And He’s Just Getting Started. As Huang hagiography goes, it was pretty good. The quote in the tweet is from Jensen Huang talking about the early days:

“We believed this model of computing could solve problems that general-purpose computing fundamentally couldn’t,” Huang says. “We also observed that video games were simultaneously one of the most computationally challenging problems and would have incredibly high sales volume. Those two conditions don’t happen very often. Video games was our killer app—a flywheel to reach large markets funding huge R&D to solve massive computational problems.”

I don't disagree with what Huang said. Despite the need to focus on gaming, we did have a vague idea that in the future there would be other application areas in which custom accelerators could make an impact. And it is true that Nvidia's VCs, Sutter Hill and Sequoia, gave us the time to develop a multi-generation architecture rather than rushing out a "minimum viable product". I do quibble with the idea that this was "genius foresight".

90s System Diagram

Even back when I was working with Curtis Priem and Chris Malachowsky on Sun's GX graphics chips the problem we were solving was that the CPU could not write pixels into the framebuffer fast enough for any kind of 3D application. So a chip, then called a graphics chip but now called a GPU, was needed between the CPU and the framebuffer capable of converting a small number of writes from the CPU into a large number of writes to the framebuffer. Designs faced three major performance constraints:

The limited bandwidth of the bus carrying instructions from the CPU to the graphics chip. I wrote about how we addressed this constraint in Engineering For The Long Term.
The limited write bandwidth of the framebuffer.
The limited transistor budget imposed by the chip's cost target.

NV1-based Diamond Edge
Swaaye, CC-By-SA 3.0

Six months after we started Nvidia, we knew of over 30 other companies all trying to build 3D graphics chips for the PC. They fell into two groups. Nvidia and a few others were making fixed-function accelerators, but most were trying to get faster time-to-market by making programmable accelerators in which the functions were implemented in software running on a CPU in the graphics chip.

One problem for the programmable companies was that the transistors needed to implement the graphic chip's CPU's fetch, decode, execute system were not available to implement the actual graphics functions, such as matrix multiply. A second problem was that the CPU in the graphics chip needed to fetch its instructions forming the program that defined the functions from some memory:

It could fetch them from the system memory that the CPU got its data and instructions from. That wasn't a good idea since (a) it required implementing a DMA engine in the graphics chip, and (b) the DMA engine's program fetches would compete for the limited bandwidth of the bus.
It could fetch them from a separate program memory private to the graphic chip. This wasn't a good idea since it added significant cost, both for the separate memory itself and also for the extra pins and complexity of the graphics chip's extra memory port.
It could fetch them from the framebuffer memory. This wasn't a good idea since the program fetches would compete with the limited bandwidth of the framebuffer RAM's random access port.

The result was that for many product generations from NV1 the winning graphics chips were all fixed-function. At the time Nvidia was started, a person having ordinary skill in the art should have understood that fixed-function accelerator hardware was the way to go. We were not geniuses.

Over the years, Moore's Law gradually relaxed the constraints, forcing the design choices to be re-evaluated. As we had expected, the one that relaxed fastest was the transistor budget. The obvious way for the accelerator to exploit the extra transistors was to perform functions in parallel. For parallel applications such as graphics this increased the accelerator advantage over the CPU. But equally important in the competitive market for graphics chips, it sped up the design by making the chip an assembly of many identical units.

The constraints continued to relax. In 2001 Nvidia had enough transistors and bandwidth to release the GeForce 3 series with programmable pixel and vertex shaders. Eventually, various non-graphics communities figured out how to use the increasing programmability of Nvidia's GPUs to accelerate their parallel applications, and Nvidia seized the opportunity to create a software platform with the 2007 release of CUDA. By greatly simplifying the development of massively parallel applications for Nvidia GPUs, CUDA drove their adoption in many fields, AI being just the latest.

Source

The Huang hagiography's focus on Nvidia's current stock price is misplaced and ahistorical. It is notoriously volatile. Look at the log plot of the stock price since the IPO. I count eight drops of 45% or more in 25 years, that's average of about one every three years. One of the questions I asked when interviewing Chris Malachowsky for the 50^th Asilomar Microcomputer Workshop was approximately "how do you manage a company with this much volatility?" His answer was, in effect, "you learn to ignore it".

Source

David Cahn of Sequoia, one of Nvidia's two VCs, writes in AI’s $600B Question:

In September 2023, I published AI’s $200B Question. The goal of the piece was to ask the question: “Where is all the revenue?”

At that time, I noticed a big gap between the revenue expectations implied by the AI infrastructure build-out, and actual revenue growth in the AI ecosystem, which is also a proxy for end-user value. I described this as a “$125B hole that needs to be filled for each year of CapEx at today’s levels.”
...
The $125B hole is now a $500B hole: In the last analysis, I generously assumed that each of Google, Microsoft, Apple and Meta will be able to generate $10B annually from new AI-related revenue. I also assumed $5B in new AI revenue for each of Oracle, ByteDance, Alibaba, Tencent, X, and Tesla. Even if this remains true and we add a few more companies to the list, the $125B hole is now going to become a $500B hole.

Goldman Sachs' provides furtehr skepticism in Gen AI: Too Much Spend, Too Little Benefit?:

Tech giants and beyond are set to spend over $1tn on AI capex in coming years, with so far little to show for it. So, will this large spend ever pay off? MIT’s Daron Acemoglu and GS’ Jim Covello are skeptical, with Acemoglu seeing only limited US economic upside from AI over the next decade and Covello arguing that the technology isn’t designed to solve the complex problems that would justify the costs, which may not decline as many expect.

Another huge drop in the stock price is sure to be in the offing. How great will the hagiographers think Huang is then?

Company	Market	Quarterly	Employees	MktCap per	Income per
	Cap	Income		Employee	Employee
NVDA	$3T	$26B	30K	$100M	$867K
GOOG	$2.3T	$25B	185K	$17M	$140K
AAPL	$3.2T	$28B	160K	$20M	$170K
MSFT	$3.3T	$22B	221K	$15M	$100K

Mr. Market will do what Mr. Market does, the stock price isn't under Huang's control. The things that are under Huang's control are the operating profit margin (53%), revenues ($26B/quarter), and the company's incredible efficiency. Nvidia's peers in the $2-3T market cap range have between 5 and 7 times as many employees. As Huang says, Nvidia is the smallest big company.

More On The Halvening

2024-07-11T08:00:00.006-07:00

At the end of May I wrote One Heck Of A Halvening about the aftermath of the halving of Bitcoin's block reward on April 19^th. Six weeks later it is time for a quick update, so follow me below the fold.

Source

All the graphs in this post are 7-day moving averages, to filter out some of the noise. The first shows what the halvening did to the hash rate. Before the halvening it was around 620EH/s. Except for a spike up to 657EH/s on 26^th May, after the BTC "price" hit nearly $71.5K, it has hovered around 580EH/s. Since power is the biggest cost, the miners' costs have dropped up to 6.5% since the halvening.

Source

The second graph is much more dramatic. It shows miners' revenue dropping from around $70M/day before the the halvening to around $30M/day after, a drop of 57%. So with costs down 6.5% and revenue down 57% their margins have taken a huge hit.

Back in May I wrote:

Lets start back in October when the Bitcoin "price" was in the high $20Ks. This was a problem, because only the most efficient miners could make a profit at that "price".

Mysteriously, that was when Tether's money printer started running at full speed, and the "price" got pumped up to around $70K. If I'm right that only the most efficient miners could make money with the "price" in the high $20Ks, and if their (income - costs) has dropped 50%, the most efficient miners need the "price" to be in the mid-$50Ks. Mysteriously, the mid-$50Ks is where the "price" has dropped to, down 21% since its last peak on June 5^th.

Source

That the hash rate was broadly flat after the halvening is interesting. Before the pre-halvening pump started, with the BTC "price" roughly flat, there was a slow but steady increase in the hash rate reflecting Bitmain's shipments of new, efficient chips displacing a smaller contribution from older chips being turned off. Bitmain's shipments continued, but once the block reward was halved many more of the less efficient chips should have been turned off, and the hash rate reduced. This didn't happen, and I can see two possible explanations:

It may be that mining power is highly skewed, with the vast majority coming from the latest chips running on low-cost power which are still profitable after the halvening. Then the shutdown of a huge number of older chips sucked in by the pre-halvening pump would have had only a small reduction in the total hash rate.
It may be that miners are burning their excess profit from the pre-halvening pump by staying in the market rather than pivoting to AI, each hoping to be among the survivors from the eventual cull.

Source

Another feature of the halvening was a massive spike in transaction fees to over $40/transaction. The third graph shows this spike, and a smaller one around the peak in the BTIC "price". But it also shows that right now no-one wants to transact, because the average fee is now $1.72. So the miners are once again almost entirely dependent upon inflating the currency with block rewards.

Source

The last graph shows the total cost of a transaction. It shows that the fee spike around the "price" peak made little difference, The current average cost per transaction is $48.57, of which $1.72 is the fee. So the system is 96.5% supported by inflation; there is currently no risk of the dangers associated with Fee-Only Bitcoin.

Engineering For The Long Term

2024-07-09T08:00:00.000-07:00

Content Warning: this post contains blatant self-promotion.

Contributions to engineering fields can only reasonably be assessed in hindsight, by looking at how they survived exposure to the real world over the long term. Four of my contributions to various systems have stood the test of time. Below the fold, I blow my own horn four times.

Four Decades

X11R1 on a Sun/1

Wikipedia has a pretty good history of the early days of the X Window System. In The X Window System At 40 I detailed my contributions to the early development of X. To my amazement 40 years after Bob Scheifler's initial release it is still valiantly resisting obsolescence. I contributed to the design, implementation, testing, release engineering and documentation of X11 starting a bit over 39 years ago. At least my design for how X handles keyboards is still the way it works.

All this while I was also working on a competitor, Sun's NeWS — which didn't survive the test of time.

Nearly Three-and-a-Half Decades

One of the things I really enjoyed about working on NeWS was that the PostScript environment it implemented was object-oriented, a legacy of PostScript's origins at Xerox PARC. Owen Densmore and I developed A User‐Interface Toolkit in Object‐Oriented PostScript that made developing NeWS applications very easy, provided you were comfortable with an object-oriented programming paradigm.

I think it was sometime in 1988 while working on the SunOS 4.0 kernel that I realized that the BSD Vnode interface was in a loose sense object-oriented. It defines the interface between the file system and the rest of the kernel. An instance of BSD's type vnode consisted of some instance data and a pointer to an "ops vector" that defined its class via an array of methods (function pointers). But it wasn't object-oriented enough to, for example, implement inheritance properly.

This flaw had led to some inelegancies as the interface had evolved through time, but what interested me more was the potential applications that would be unleashed if the interface could be made properly object-oriented. Instead of being implemented from scratch, file systems could be implemented by sub-classing other file systems. For example, a read-only file system such as a CD-ROM could be made writable by "stacking" a cache file system on top, as shown in Figure 11. I immediately saw the possibility of significant improvements in system administration that could flow from stacking file systems.

Evolving the Vnode Interface: Fig. 11

I started building a prototype by performing major surgery on a copy of the code that would become SunOS 4.1. By late 1989 it worked well enough to demonstrate the potential of the idea, so I published 1990's Evolving the Vnode Interface. The paper describes a number of Vnode modules that can be stacked together to implement interesting functions. Among them was cache-fs, which layered a writable local file system above a local or remote read-only file system:

This simple module can use any file system as a file-level cache for any other (read-only) file system. It has no knowledge of the file systems it is using; it sees them only via their opaque vnodes. Figure 11 shows it using a local writable ufs file system to cache a remote read-only NFS file system, thereby reducing the load on the server. Another possible configuration would be to use a local writable ufs file system to cache a CD-ROM, obscuring the speed penalty of CD.

Over the next quarter-century the idea of stacking vnodes and the related idea of "union mounts" from Rob Pike and Plan 9 churned around until, in October 2014, Linus Torvalds added overlayfs to the 3.18 kernel. I covered the details of this history in 2015's It takes longer than it takes. In it I quoted from Valerie Aurora's excellent series of articles about the architectural and implementation difficulties involved in adding union mounts to the Linux kernel. I concurred with her statement that:

The consensus at the 2009 Linux file systems workshop was that stackable file systems are conceptually elegant, but difficult or impossible to implement in a maintainable manner with the current VFS structure. My own experience writing a stacked file system (an in-kernel chunkfs prototype) leads me to agree with these criticisms.

I wrote:

Note that my original paper was only incidentally about union mounts, it was a critique of the then-current VFS structure, and a suggestion that stackable vnodes might be a better way to go. It was such a seductive suggestion that it took nearly two decades to refute it!

Nevertheless, the example I used in Evolving the Vnode Interface of a use for stacking vnodes was what persisted. It took a while for the fact that overlayfs was an official part of the Linux kernel to percolate through the ecosystem, but after six years I was able to write Blatant Self-Promotion about the transformation it wrought on Linux's packaging and software distribution, inspired by Liam Proven's NixOS and the changing face of Linux operating systems. He writes about less radical ideas than NixOS:

So, instead of re-architecting the way distros are built, vendors are reimplementing similar functionality using simpler tools inherited from the server world: containers, squashfs filesystems inside single files, and, for distros that have them, copy-on-write filesystems to provide rollback functionality.

The goal is to build operating systems as robust as mobile OSes: periodically, the vendor ships a thoroughly tested and integrated image which end users can't change and don't need to. In normal use, the root filesystem is mounted read-only, and there's no package manager.

Since then this model has become universal. Distros ship as a bootable ISO image, which uses overlayfs to mount a writable temporary file system on top. This is precisely how my 1989 prototype was intended to ship SunOS 4.1. The technology has spread to individual applications with systems such as Snaps and Flatpak.

Three Decades

The opportunity we saw when we started Nvidia was that the PC was transitioning from the ISA bus to version 1 of the PCI bus. The ISA bus' bandwidth was completely inadequate for 3D games, but the PCI bus had considerably more. Whether it was enough was an open question. We clearly needed to make the best possible use of the limited bandwidth we could get.

Nvidia's first chip had three key innovations:

Rendering objects with quadric patches not triangles. A realistic model using quadric patches needed perhaps a fifth of the data for an equivalent triangle model.
I/O virtualization with applications using a write-mostly, object-oriented interface. Read operations are neccessarily synchronous, whereas write operations are asynchronous. Thus the more writes per read across the bus, the better the utilization of the available bus bandwidth.
A modular internal architecture based on an on-chip token-ring network. Thie goal was that each functional unit be simple enough to be designed and tested by a three-person team.

SEGA's Virtua Fighter on NV1

The first two of these enabled us to get Sega's arcade games running at full frame rate on a PC. Curtis Priem and I designed the second of these, and it is the one that has lasted:

I/O virtualization allowed multiple processes direct access to the graphics hardware, with no need to pass operations through the operating system. I explained the importance of this a decade ago in Hardware I/O Virtualization, using the example of Amazon building their own network interface cards. Tne first chip appeared on the bus as having 128 wide FIFOs. The operating system could map one of them into each process wanting access to the chip, allowing applications direct access to the hardware but under the control of the operating system.
The interface was write-mostly because the application could read from the FIFO the number of free slots, that is the number of writes before the bus would stall.
The interface was object-oriented because the data and the offset in the FIFO formed an invocation of a method on an instance of a (virtual) class. Some classes were implemented in hardware, others trapped into the kernel and were implemented by the driver, but the application just created and used instances of the available classes without knowing which was which. The classes were arranged in a hierarchy starting with class CLASS. Enumerating the instances of class CLASS told the application which classes it could use. Enumerating the instances of each of those classes told the application how many of each type of resource it could use.

The importance of the last of these was that it decoupled the hardware and software release schedules. Drivers could emulate classes that had yet to appear in hardware, the applications would use the hardware once it was available. Old software would run on newer hardware, it would just see some classes it didn't know how to use. One of our frustrations with Sun was the way software and hardware release schedules were inextricably interlinked.

Two-and-a-Half Decades

Last October I celebrated the LOCKSS Program Turns 25. Vicky Reich explained to me how libraries preserved paper academic journals and how their move to the Web was changing libraries role from purchasing a copy of the journal to renting access to the publisher's copy, and I came up with the overall peer-to-peer archtecture (and the acronym). With help from Mark Seiden I built the prototype, and after using it to demonstrate the feasibility of the concept, also used it to show vulnerabilities in the initial protocol. In 2003 I was part of the team that solved these problems, for which we were awarded the Best Paper award at the Symposium on Operating System Principles for Preserving peer replicas by rate-limited sampled voting.

X Window System At 40

2024-07-02T08:00:00.000-07:00

X11R1 on Sun
Techfury90 CC0

I apologize that this post is a little late. On 19^th June the X Window System celebrated its 40th birthday. Wikipedia has a comprehensive history of the system including the e-mail Bob Scheifler sent announcing the first release:

From: rws@mit-bold (Robert W. Scheifler)
To: window@athena
Subject: window system X
Date: 19 June 1984 0907-EDT (Tuesday)

I've spent the last couple weeks writing a window
system for the VS100. I stole a fair amount of code
from W, surrounded it with an asynchronous rather
than a synchronous interface, and called it X. Overall
performance appears to be about twice that of W. The
code seems fairly solid at this point, although there are
still some deficiencies to be fixed up.

We at LCS have stopped using W, and are now
actively building applications on X. Anyone else using
W should seriously consider switching. This is not the
ultimate window system, but I believe it is a good
starting point for experimentation. Right at the moment
there is a CLU (and an Argus) interface to X; a C
interface is in the works. The three existing
applications are a text editor (TED), an Argus I/O
interface, and a primitive window manager. There is
no documentation yet; anyone crazy enough to
volunteer? I may get around to it eventually.

Anyone interested in seeing a demo can drop by
NE43-531, although you may want to call 3-1945
first. Anyone who wants the code can come by with a
tape. Anyone interested in hacking deficiencies, feel
free to get in touch.

Scheifler was right that it was a "good starting point for experimentation", but it wasn't really a usable window system until version 11 was released on 15^th September 1987. I was part of the team that burned the midnight oil at MIT to get that release out, but my involvement started in late 1985.

Below the fold are some reflections on my contributions, some thoughts on the astonishing fact that the code is still widely deployed after 40 years, and some ideas on why it has been so hard to replace.

Involvement

I arrived at Sun in September 1985 to work with James Gosling on a window system for Sun's workstations. We had worked together developing the window system for Carnegie-Mellon's Andrew project. We both realized that, if Unix and Unix workstations like Sun's were to succeed, they had to have a widely adopted window system. Ideally, we thought, it would have three key attributes:

An imaging model at a higher level than simply a grid of pixels, so that applications didn't have to adapt to varying screen resolutions. The clear favorite was Adobe PostScript's, which had appeared on the Apple LaserWriter in January 1985.
Networking, so that applications could run wherever it made sense with their user interface where the user was. In the mid-80s both workstations and networks were slow, running a window system was a big load on a Motorola 68000. Having two CPUs working on an application, the client-server model, helped.
Programmability, because it was hard to provide a good user experience if there was a slow network round-trip required for each response to a user action. If the window system were programmable, the code that responded to the user's actions could run on the user's workstation, eliminating the network round-trip.

Gosling had left C-MU some months earlier. He was adamant that it was possible to implement PostScript on a 68000 fast enough to be a window system. Adding networking, mouse and keyboard extensions to PostScript would satisfy all three of our requirements. Because Adobe's PostScript in the 68000-based LaserWriter was notoriously slow I was skeptical, but I should have known better than to doubt his coding skills. A couple of months later when Gosling showed me a prototype PostScript interpreter running incredibly fast on a Sun/1, I was convinced to join him on what became the NeWS project.

Once at Sun I realized that it was more important for the company that the Unix world standardized on a single window system than that the standard be Sun's NeWS system. At C-MU I had already looked into X as an alternative to the Andrew window system, so I knew it was the obvious alternative to NeWS. Although most of my time was spent developing NeWS, I rapidly ported X version 10 to the Sun/1, likely the second port to non-DEC hardware. It worked, but I had to kludge several areas that depended on DEC-specific hardware. The worst was the completely DEC-specific keyboard support.

Because it was clear that a major redesign of X was needed to make it portable and in particular to make it work well on Sun hardware, Gosling and I worked with the teams at DEC SRC and WRL on the design of X version 11. Gosling provided significant input on the imaging model, and I designed the keyboard support. As the implementation evolved I maintained the Sun port and did a lot of testing and bug fixing. All of which led to my trip to Boston to pull all-nighters at MIT finalizing the release.

My involvement continued after the first release. I was the editor and main author of the X Inter-Client Communications Conventions Manual (ICCCM) that forms Part III of Robert Scheifler and Jim Gettys' X Window System. A user's X environment consists of a set of client applications, an X server managing displays and input devices, and a window manager application implementing a user interface that allows the user to allocate the server's resources such as windows to the clients. The window manager needs to communicate the resource allocations to the clients, and the clients need to communicate between themselves to support, for example, cut-and-paste. The ICCCM explains its scope thus:

Being a good citizen in the X Version 11 world involves adhering to conventions that govern inter-client communications in the following ares:

Selection mechanism

Cut buffers

Window manager

Session manager

Manipulation of shared resources

Device color characterization

This part of the book proposes suitable conventions without attempting to enforce any particular user interface.

xkcd 2730

There is a certain justice in The UNIX-HATERS Handbook's description of my efforts:

one of the most amazing pieces of literature to come out of the X Consortium is the “Inter Client Communication Conventions Manual,” ... It describes protocols that X clients must use to communicate with each other via the X server, including diverse topics like window management, selections, keyboard and colormap focus, and session management. In short, it tries to cover everything the X designers forgot and tries to fix everything they got wrong. But it was too late—by the time ICCCM was published, people were already writing window managers and toolkits, so each new version of the ICCCM was forced to bend over backwards to be backward compatible with the mistakes of the past.

Jim Morris, who ran the Andrew project at C-MU, was very wise about what we were getting into: "We're either going to have a disaster, or a success disaster". He often added: "And I know which I prefer!". Jim was exaggerating to make two points:

Being a success didn't mean the problems were over, it meant a different set of problems.
Ignoring the problems of success was a good way of preventing it.

X11 was definitely a "success disaster".

Why X not NeWS?

NeWS Xardox
Public Domain

NeWS was amazing technology, providing each of our requirements with usable performance on a 68000. Its interpreter was a clean-room implementation of PostScript from the "~~Blue~~Red Book", with extensions that provided cooperative multi-threading, networking, arbitrary-shaped translucent windows, and a fully object-oriented toolkit. Steve Jobs agreed with us that PostScript was the right imaging model, although the Display PostScript interpreter he licensed from Adobe for the NextSTEP window system was far more restricted. The overwhelming success of JavaScript has validated our argument for programmability. So why did X11 take over the world? My view is that there are three main arguments, any one of which would have been decisive:

We and Jobs were wrong about the imaging model, for at least two reasons. First, early on pixels were in short supply and applications needed to make the best use of the few they were assigned. They didn't want to delegate control to the PostScript interpreter. Second, later on came GPUs with 3D imaging models. The idea of a one-size-fits-all model became obsolete. The reason that Wayland should replace X11 is that it is agnostic to the application's choice of imaging model.
Although we were right about the need for programmability, PostScript was the wrong language. Even back in 1985 Gosling and I were veteran programmers, with experience in many languages including not just C and Assembler, but also LISP, Prolog and Smalltalk. We didn't find PostScript's stack-based language intimidating. But already by that time Darwinian evolution had focused CS education on C-like languages, so the majority of application programmers were intimidated. And NeWS wasn't just stack-oriented, it was also object-oriented and multi-threaded, and none of these were optional, making it even more intimidating. In a sense NeWS was an echo of the success of Sun in the early days — a bunch of really good engineers building the system they wanted to use, not one for the mass market.
A major reason for Sun's early success was that they in effect open-sourced the Network File System. X11 was open source under the MIT license. I, and some of the other Sun engineers, understood that NeWS could not displace X11 as the Unix standard window system without being equally open source. But Sun's management looked at NeWS and saw superior technology, an extension of the PostScript that Adobe was selling, and couldn't bring themselves to give it away. But they also couldn't ignore the fact that X11 was popular with their customer base, who could run the MIT X11 environment any time they wanted. The result was the ghastly kludge called the Xnews server, a monument to the inability of Sun to follow McNealy's "all the wood behind one arrow" dictum. Wikipedia correctly notes that:

This seriously degraded the NeWS interpreter performance and was not considered a very good X11 server either.

Longevity

The anniversary bought forth several commentaries on its extraordinary longevity, including:

Richard Speed's The X Window System is still hanging on at 40 is aptly subtitled "Never underestimate the stickiness of legacy technology" and concludes:

In 2022, we wondered if "Wayland has what it takes to replace X." Two years later, the question is still open, although the direction of travel is clear. Yet the stickiness of "it just works" is not to be underestimated, and we would not be surprised if the 50th anniversary rolls around and there is still someone clinging to X11 for that one old app that won't run properly on anything else.
This is very likely. In order to displace X11, Wayland had to be compatible with it via the XWayland server. So there is little motivation to rewrite existing applications to use Wayland directly.
Kevin Purdy's 40 years later, X Window System is far more relevant than anyone could guess starts:

Often times, when I am researching something about computers or coding that has been around a very long while, I will come across a document on a university website that tells me more about that thing than any Wikipedia page or archive ever could.

It's usually a PDF, though sometimes a plaintext file, on a .edu subdirectory that starts with a username preceded by a tilde (~) character. This is typically a document that a professor, faced with the same questions semester after semester, has put together to save the most time possible and get back to their work. I recently found such a document inside Princeton University's astrophysics department: "An Introduction to the X Window System," written by Robert Lupton.

X Window System, which turned 40 years old earlier this week, was something you had to know how to use to work with space-facing instruments back in the early 1980s, when VT100s, VAX-11/750s, and Sun Microsystems boxes would share space at college computer labs. As the member of the AstroPhysical Sciences Department at Princeton who knew the most about computers back then, it fell to Lupton to fix things and take questions.
I really like Lupton's assessment:

"It worked, at least relative to the other options we had," Lupton said. He noted that Princeton's systems were not "heavily networked in those days," such that the network traffic issues some had with X weren't an issue then. "People weren't expecting a lot of GUIs, either; they were expecting command lines, maybe a few buttons... it was the most portable version of a window system, running on both a VAX and the Suns at the time... it wasn't bad."
Purdy quotes Evan Jenkins from thirteen years ago:

X is the oldest lady at the dance, and she insists on dancing with everyone. X has millions of lines of source, but most of it was written long ago, when there were no GPUs, and no specialized transistors to do programmable shading or rotation and translation of vertexes. The hardware had no notion of oversampling and interpolation to reduce aliasing, nor was it capable of producing extremely precise color spaces. The time has come for the old lady to take a chair.

There hasn't been an official release of X11 for twelve years, but it is proving very hard to kill off.