Titan in depth: Security in plaintext

While there are no absolutes in computer security, we design, build and operate Google Cloud Platform (GCP) with the goal to protect customers' code and data. We harden our architecture at multiple layers, with components that include Google-designed hardware, a Google-controlled firmware stack, Google-curated OS images, a Google-hardened hypervisor, as well as data center physical security and services.

In this post, we provide details of the mechanisms of how we will establish a hardware root of trust using our custom chip, Titan.

First introduced at Google Cloud Next '17, Titan is a secure, low-power microcontroller designed with Google hardware security requirements and scenarios in mind. Let’s take a look at how Titan works to ensure that a machine boots from a known good state using verifiable code, and establishes the hardware root of trust for cryptographic operations in our data centers.

Machine boot basics 

1. The machine's boot process starts when the BMC configuring the machine hardware lets the CPU come out of reset. 
2. The CPU then loads the basic firmware (Boot or UEFI) from the boot firmware flash, which performs further hardware/software configuration. 
3. Once the machine is sufficiently configured, the boot firmware accesses the "boot sector" on the machine's persistent storage, and loads a special program called the "boot loader" into the system memory. 
4. The boot firmware then passes execution control to the boot loader, which loads the initial OS image from storage into system memory and passes execution control to the operating system. 

As privileged software attacks increase and more research becomes available on rootkits, we have committed to delivering secure boot and hardware-based root of trust for machines that form our infrastructure and host our Google Cloud workloads.

Secure boot with Titan 

To achieve these security properties, Titan comprises several components: a secure application processor, a cryptographic co-processor, a hardware random number generator, a sophisticated key hierarchy, embedded static RAM (SRAM), embedded flash and a read-only memory block. Titan communicates with the main CPU via the Serial Peripheral Interface (SPI) bus, and interposes between the boot firmware flash of the first privileged component, e.g., the BMC or Platform Controller Hub (PCH), allowing Titan to observe every byte of boot firmware.

Titan's application processor immediately executes code from its embedded read-only memory when its host machine is powered up. The fabrication process lays down immutable code, known as the boot ROM, that is trusted implicitly and validated at every chip reset. Titan runs a Memory Built-In Self-Test every time the chip boots to ensure that all memory (including ROM) has not been tampered with. The next step is to load Titan’s firmware. Even though this firmware is embedded in the on-chip flash, the Titan boot ROM does not trust it blindly. Instead, the boot ROM verifies Titan's firmware using public key cryptography, and mixes the identity of this verified code into Titan's key hierarchy. Then, the boot ROM loads the verified firmware.

Once Titan has booted its own firmware in a secure fashion, it will turn its attention to the host’s boot firmware flash, and verify its contents using public key cryptography. Titan can gate PCH/BMC access to the boot firmware flash until after it has verified the flash content, at which point it signals readiness to release the rest of the machine from reset. Holding the machine in reset while Titan cryptographically verifies the boot firmware provides us the first-instruction integrity property: we know what boot firmware and OS booted on our machine from the very first instruction. In fact, we even know which microcode patches may have been fetched before the boot firmware's first instruction. Finally, the Google-verified boot firmware configures the machine and loads the boot loader, which subsequently verifies and loads the operating system.

Cryptographic identity using Titan 

The Titan-based identity system not only verifies the provenance of the chips creating the CSRs, but also verifies the firmware running on the chips, as the code identity of the firmware is hashed into the on-chip key hierarchy. This property enables remediation and allows us to fix bugs in Titan firmware, and issue certificates that can only be wielded by patched Titan chips. The Titan-based identity system enables back-end systems to securely provision secrets and keys to individual Titan-enabled machines, or jobs running on those machines. Titan is also able to chain and sign critical audit logs, making those logs tamper-evident. To offer tamper-evident logging capabilities, Titan cryptographically associates the log messages with successive values of a secure monotonic counter maintained by Titan, and signs these associations with its private key. This binding of log messages with secure monotonic counter values ensures that audit logs cannot be altered or deleted without detection, even by insiders with root access to the relevant machine.

Conclusion 

In short:

1. Titan provides a hardware-based root of trust that establishes strong identity of a machine, with which we can make important security decisions and validate the “health” of the system. 
2. Titan offers integrity verification of firmware and software components. 
3. The system’s strong identity ensures that we'll have a non-repudiable audit trail of any changes done to the system. Tamper-evident logging capabilities help identify actions performed by an insider with root access.