はてなブックマーク

In our pursuit of becoming a better full service research firm, we’ve moved off Substack. For any questions please read https://semianalysis.com/faq/#substack There has been an increasing amount of fear, uncertainty and doubt (FUD) regarding AI Scaling laws. A cavalcade of part-time AI industry prognosticators have latched on to any bearish narrative they can find, declaring the end of scaling law

Groq Inference Tokenomics: Speed, But At What Cost? Faster than Nvidia? Dissecting the economics Faster than Nvidia? Dissecting the economicsGroq, an AI hardware startup, has been making the rounds recently because of their extremely impressive demos showcasing the leading open-source model, Mistral Mixtral 8x7b on their inference API. They are achieving up to 4x the throughput of other inference

Demystifying GPT-4: The engineering tradeoffs that led OpenAI to their architecture. OpenAI is keeping the architecture of GPT-4 closed not because of some existential risk to humanity but because what they’ve built is replicable. In fact, we expect Google, Meta, Anthropic, Inflection, Character, Tencent, ByteDance, Baidu, and more to all have models as capable as GPT-4 if not more capable in the

Google “We Have No Moat, And Neither Does OpenAI” Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAI Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAI The text below is a very recent leaked document, which was shared by an anonymous individual on a public Discord server who has granted permission for its republication. It o

Google OCS Apollo: The >$3 Billion Game-Changer in Datacenter Networking Custom Optical Switches Reduce Broadcom's Networking Dominance Custom Optical Switches Reduce Broadcom’s Networking Dominance Networking is a critical part of any datacenter, especially with the rise of networking-intensive large language models. As such, it was a clear target for Google’s infrastructure optimization efforts.

How Nvidia’s CUDA Monopoly In Machine Learning Is Breaking – OpenAI Triton And PyTorch 2.0 Over the last decade, the landscape of machine learning software development has undergone significant changes. Many frameworks have come and gone, but most have relied heavily on leveraging Nvidia’s CUDA and performed best on Nvidia GPUs. However, with the arrival of PyTorch 2.0 and OpenAI’s Triton, Nvidia’

Arm is widely regarded as the most important semiconductor IP firm. Their IP ships in billions of new chips every year from phones, cars, microcontrollers, Amazon servers, and even Intel’s latest IPU. Originally it was a British owned and headquartered company, but SoftBank acquired the firm in 2016. They proceeded to plow money into Arm Limited to develop deep pushes into the internet of things,