Exclusive: Top execs explain Apple’s chip philosophy: ‘We are not a merchant silicon company, trying to leave nothing on the table’

Tom Boger, Vice President of Mac Product Marketing at Apple, who joined Millet on the call, underlined that it was unusual to see this “pace of innovation year after year after year” when it comes to silicon. The first M-series chip was released just four years ago. “That is the promise. That is a commitment we make to our teams to deliver innovations as they are available to us,” he said.

Apple’s secret weapon

Explaining what gives Apple silicon an edge now, Millet said, “We take advantage of the three major components, the architecture, the design, and the process technology. Our fourth tool, really our secret weapon, I think, is our ability to co-design these amazing chips with the system teams and the product designers as they are imagining possibilities.”

The display engine of M4 is enhanced to support two external displays in addition to a built-in display. (Image source: Apple)

Millet cited the new Mac mini as a perfect example of that. “The opportunity was for us and for the design team to be able to come together and build this incredible new platform… there is no way that machine could have come to life without that collaboration. And that is really what Apple is all about,” he said.

Always designed for AI

With Cupertino starting to roll out Apple Intelligence features in different parts of the world, Boger clarified that they have “always had intelligent features in our Mac”. “One of the things that we announced is the fact that Apple Intelligence runs on any Mac with an M series chip, right? So from the first M1 chip, we have included the Neural Engine, and we have a great architecture for AI. And we also have developers taking advantage of Apple silicon to offer our customers intelligent features. So the M Series chips were always built for AI,” he said.

But what made Apple prepare for this AI era in advance, especially when the real focus started picking up only after OpenAI announced ChatGPT in 2022?

“How did we know? I think we have to go back to 2017 when we introduced the Neural Engine in our iOS products. And really this was inspired by our recognition of the importance of computational photography,” explained Millet. “So we were seeing the amazing research that folks up in the University of Toronto were demonstrating… these new neural networks were capable of doing image recognition beyond the capacity of humans, or at least matching, and they were headed on a trajectory that was clear. And so we pounced on the opportunity to build that embedded capability into our camera processors for the phone,” he added.

But the folks at Apple realised this trend was just starting. “We knew we wanted to take our Neural Engine, centralise it, and make it a first-class module within the SOC. We build these SOCs every year and we have the opportunity to pick up the latest research, the latest direction,” Millet added.

He recalled how an “interesting paper” was published in 2017 that led to the invention of the transformer network, which then became the foundation for the LLMs that exploded in 2022. “It took about five years. Folks on my team recognised that these papers had the potential to be very interesting and might have a huge impact and made big architectural changes to the Neural Engine, just in case. We introduced the first transformer-capable SOCs in 2020 when we introduced the M1 and this all lined up perfectly,” he said.

Millet accepts that it is a “little bit of good fortune that this investment turned out to pay off”. “On the other hand, it shows you the diligence that we spend all our time trying to figure out where the ball is moving. We try to make sure we are there before it gets there,” he added.

‘Just push things forward’

“One of the things that we do with Apple silicon is we advance the state of the art, right? And we push things forward, whatever metric you measure it by, whether it is CPU performance, GPU performance, Neural Engine performance because that has always been the name of the game,” added Boger, underlining that Apple silicon continues to just push things further “because that’s what our customers do”.

Generation after generation of the A Series chips made for iPhones and iPads, Boger said, had this tremendous focus on power efficiency, and they have kept that focus with the M series chips which are now in its fourth generation. “No other platform can touch our power performance per watt. That is the tangible benefit to users; for instance, you take that MacBook Pro as an example, and you run the most demanding workload while you have it plugged in, and then you unplug it…it is going to give you the exact same performance,” he added.

The difference between M4 Max and M4 Pro

Millet explained that when it comes to running LLMs, the big difference between the M4 Max and the M4 Pro is the memory system. “M4 Max has effectively about twice the memory bandwidth of M4 Pro. The responsiveness will come from the Neural Engine which is present in both so the compute side of it will be balanced and equal. The memory bandwidth side of it will help M4 Max for some of the bigger models to hit higher token rates. So the more complex models, actually could, in theory, run a little bit faster,” he elaborated.

M4 Max features a 16-core CPU that’s up to 2.2x faster than the CPU in M1 Max, and up to 2.5x faster than the latest AI PC chip. (Image source: Apple)

But it is not that simple. “Now, truth be told… these systems are so fast it is hard to tell the difference. But for someone who was really pushing the edge for a very, very large model, they will definitely benefit from that wider memory system on the Max,” Millet said.

A healthy obsession

Millet said the obsession within the silicon team under Johny Srouji, Apple’s Senior Vice President of Hardware Technologies, is to increase the arc ISO power and that is how they judge themselves. For the layman, this means that when running the same software on the M3 Max and M4 Max, the workload does not change or it does not burn more energy. However, Apple silicon is delivering the exact same performance, generation after generation, at dramatically lower power.

“So for example, let us say a particular workload is running 20 per cent faster at the same power. Well, it might be running at 30 per cent less energy at the same performance,” Millet explained. And this is why Apple devices powered by M-series processors can offer more battery life than the competition even as they get faster and more powerful every year.

The Mac mini’s superpower

This also manifests in a device like the tiny new Mac mini which can for all practical purposes replace an Apple Studio computer used for video and other heavy workloads. Boger is clear this is a testament to Apple silicon. “It is a superpower, it is performance per watt. And we don’t make a bunch of chips and then decide where we are going to put them. We design our chips from the ground up for our products, and that is a tremendous strategic advantage that we have,” Boger said.

Focusing on workloads

But is it getting tougher to push the envelope when it comes to processors?

“You can be confident we are looking under every rock and looking at every opportunity,” Millet countered, adding, “And, we have fortunately, some amazing people on our teams who have lots of really interesting creative ideas. And when we are looking for ideas around the process technology, we have experts in the field, our silicon partners. When we are looking at architectural ideas, we have some of the best modelling and conceptual computer scientists on the planet, and we work closely with our software partners to look for all the best opportunities to accelerate not just generic benchmarks, which we often get judged by, but more importantly by the workloads that we are actually delivering to our customers.”

The new MacBook Pro with M4 Pro speeds up a variety of workflows such as structural engineering, data modeling, and more. (Image source: Apple)

He clarified that it is not as if they do not pay attention to the benchmarks. “But I tell my team, focus on the Apple workloads first, and then we will see how those benchmarks work out,” Miller said.

Millet reiterated that a lot of what Apple silicon is able to achieve is because they are vertically integrated and they know what software Apple is going to be trying to put on these platforms. “We know what the hardware system and thermal design will look like, and we understand what the process technology nodes are, and we aggressively pursue our best silicon options… I have been doing this for more than 30 years — it is the best situation to be in,” Miller said.