This page describes how to use manually-created customer-managed encryption keys (CMEK) for Spanner.
To learn more about CMEK, see the Customer-managed encryption keys (CMEK) overview.
Create a CMEK-enabled database
-
Create a key in Cloud Key Management Service (Cloud KMS). Spanner supports creating to following Cloud KMS types:
The key must be in the same location as your Spanner instance. For example, if your Spanner instance configuration is in
us-west1
, then your Cloud KMS key ring location must also beus-west1
.Not every Spanner multi-region instance configuration has a corresponding Cloud KMS key ring location. For Spanner databases in custom, dual-region, or multi-region instance configurations, you can use multiple regional (single-region) Cloud KMS keys to protect your database. For example:
- If your Spanner database is in the multi-region instance
configuration
nam14
, then you can create Cloud KMS keys inus-east4
,northamerica-northeast1
, andus-east1
. - If your database is in a custom instance configuration that uses
nam3
as the base instance configuration with an additional read-only replica inus-central2
, then you can create Cloud KMS keys inus-east4
,us-east1
,us-central1
, andus-central2
.
Optional: To see a list of the replica locations in your Spanner instance configuration, use the
gcloud spanner instances get-locations
command:gcloud spanner instances get-locations <var>INSTANCE_ID</var>
For more information, see the following resources:
- If your Spanner database is in the multi-region instance
configuration
Grant Spanner access to the key.
In Cloud Shell, create and display the service agent, or display it if the account already exists:
gcloud beta services identity create --service=spanner.googleapis.com \ --project=PROJECT_ID
If you're prompted to install the gcloud Beta Commands component, type
Y
. After installation, the command is automatically restarted.The
gcloud services identity
command creates or gets the service agent that Spanner can use to access the Cloud KMS key on your behalf.The service account ID is formatted like an email address:
Service identity created: [email protected]
Grant the Cloud KMS CryptoKey Encrypter/Decrypter(
cloudkms.cryptoKeyEncrypterDecrypter
) role to the service account for each region (--location
) in your Spanner instance configuration. To do so, run thegcloud kms keys add-iam-policybinding
command:gcloud kms keys add-iam-policy-binding KMS_KEY \ --location KMS_KEY_LOCATION \ --keyring KMS_KEY_RING \ --project=PROJECT_ID \ --member serviceAccount:[email protected] \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter
Here's an example output:
Updated IAM policy for key [KMS_KEY]
If you're using multiple Cloud KMS keys to protect your database, run the
gcloud kms keys add-iam-policybinding
command for all the your keys.This role ensures that the service account has permission to both encrypt and decrypt with the Cloud KMS key. For more information, see Cloud KMS permissions and roles.
Create the database and specify your Cloud KMS key.
Console
Use the console to create databases in regional instance configurations.
In the Google Cloud console, go to the Instances page.
Click the instance where you want to create a database.
Click Create database and fill out the required fields.
Click Show encryption options.
Select Cloud KMS key.
Select a key from the drop-down list.
The list of keys is limited to the current Google Cloud project. To use a key from a different Google Cloud project, create the database using gcloud CLI instead of the Google Cloud console.
Once the database is created, you can verify that the database is CMEK-enabled by viewing the Database overview page.
gcloud
To create a CMEK-enabled database in a regional, custom, or multi-region
instance configuration, run the gcloud spanner databases create
command:
gcloud spanner databases create DATABASE \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID \
--ddl="CREATE TABLE Users (Id INT64 NOT NULL, FirstName STRING(100) NOT NULL, LastName STRING(100) NOT NULL,) PRIMARY KEY (Id)" \
--kms-project=KMS_PROJECT_ID \
--kms-location=KMS_KEY_LOCATION \
--kms-keyring=KMS_KEYRING \
--kms-keys=KMS_KEY_1[, KMS_KEY_2 ... ]
To verify that a database is CMEK-enabled, run the
gcloud spanner databases describe
command:
gcloud spanner databases describe DATABASE \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID
CMEK-enabled databases include a field for encryptionConfig
, as shown in the
following example output:
encryptionConfig:
kmsKeyNames:projects/my-kms-project/locations/eur5/keyRings/my-kms-key-ring/cryptoKeys/my-kms-key
name: projects/my-spanner-project/instances/my-instance/databases/my-db
state: READY
Client libraries
C#
To create a CMEK-enabled database in a regional instance configuration:
C++
To create a CMEK-enabled database in a regional instance configuration:
Go
To create a CMEK-enabled database in a regional instance configuration:
Java
To create a CMEK-enabled database in a regional instance configuration:
Node.js
To create a CMEK-enabled database in a regional instance configuration:
To create a CMEK-enabled database in a multi-region instance configuration:
PHP
To create a CMEK-enabled database in a regional instance configuration:
To create a CMEK-enabled database in a multi-region instance configuration:
Python
To create a CMEK-enabled database in a regional instance configuration:
Ruby
To create a CMEK-enabled database in a regional instance configuration:
To create a CMEK-enabled database in a multi-region instance configuration:
View the key versions in use
The database's encryption_info
field shows information about key versions.
When a database's key version changes, the change isn't immediately propagated
to encryption_info
. There might be a delay before the change is reflected in
this field.
Console
In the Google Cloud console, go to the Instances page.
Click the instance containing the database you want to view.
Click the database.
Encryption information is displayed on the Database details page.
gcloud
You can get a database's encryption_info
by running the
gcloud spanner databases describe
or gcloud spanner databases list
command. For example:
gcloud spanner databases describe DATABASE \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID
Here's an example output:
name: projects/my-project/instances/test-instance/databases/example-db
encryptionInfo:
- encryptionType: CUSTOMER_MANAGED_ENCRYPTION
kmsKeyVersion: projects/my-kms-project/locations/my-kms-key1-location/keyRings/my-kms-key-ring1/cryptoKeys/my-kms-key1/cryptoKeyVersions/1
- encryptionType: CUSTOMER_MANAGED_ENCRYPTION
kmsKeyVersion: projects/my-kms-project/locations/my-kms-key2-location/keyRings/my-kms-key-ring2/cryptoKeys/my-kms-key2/cryptoKeyVersions/1
Disable the key
Disable the key version(s) that are in use by following these instructions for each key version.
Wait for the change to take effect. Disabling a key can take up to three hours to propagate.
To confirm that the database is no longer accessible, execute a query in the CMEK-disabled database:
gcloud spanner databases execute-sql DATABASE \ --project=SPANNER_PROJECT_ID \ --instance=INSTANCE_ID \ --sql='SELECT * FROM Users'
The following error message appears:
KMS key required by the Spanner resource is not accessible.
Enable the key
Enable the key versions that are in use by the database by following these instructions for each key version.
Wait for the change to take effect. Enabling a key can take up to three hours to propagate.
To confirm that the database is no longer accessible, execute a query in the CMEK-enabled database:
gcloud spanner databases execute-sql DATABASE \ --project=SPANNER_PROJECT_ID \ --instance=INSTANCE_ID \ --sql='SELECT * FROM Users'
If the change has taken effect, the command executes successfully.
Back up a database
You can use Spanner backups to create backups of your databases. By default, Spanner backups created from a database use the same encryption configuration as the database itself. You can optionally specify a different encryption configuration for a backup.
Console
Use the console to create backups in regional instance configurations.
In the Google Cloud console, go to the Instances page.
Click the instance name that contains the database that you want to back up.
Click the database.
In the navigation pane, click Backup/Restore.
In the Backups tab, click Create backup.
Enter a backup name and select an expiration date.
Optional: Click Show encryption options.
a. If you want to use a different encryption configuration for your backup, click the slider next to Use existing encryption.
a. Select Cloud KMS key.
a. Select a key from the drop-down list.
The list of keys is limited to the current Google Cloud project. To use a key from a different Google Cloud project, create the database using gcloud CLI instead of the Google Cloud console.
Click Create.
The Backups table displays encryption information for each backup.
gcloud
To create a CMEK-enabled backup in a regional, custom, or multi-region
instance configuration, run the gcloud spanner backups create
command:
gcloud spanner backups create BACKUP \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID \
--database=DATABASE \
--retention-period=RETENTION_PERIOD \
--encryption-type=customer_managed_encryption \
--kms-project=KMS_PROJECT_ID \
--kms-location=KMS_KEY_LOCATION \
--kms-keyring=KMS_KEY_RING \
--kms-keys=KMS_KEY_1[, KMS_KEY_2 ... ]
--async
To verify that the backup created is CMEK encrypted:
gcloud spanner backups describe BACKUP \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID
Client libraries
C#
To create a CMEK-enabled backup in a regional instance configuration:
C++
To create a CMEK-enabled backup in a regional instance configuration:
Go
To create a CMEK-enabled backup in a regional instance configuration:
Java
To create a CMEK-enabled backup in a regional instance configuration:
Node.js
To create a CMEK-enabled backup in a regional instance configuration:
To create a CMEK-enabled backup in a multi-region instance configuration:
PHP
To create a CMEK-enabled backup in a regional instance configuration:
To create a CMEK-enabled backup in a multi-region instance configuration:
Python
To create a CMEK-enabled backup in a regional instance configuration:
Ruby
To create a CMEK-enabled backup in a regional instance configuration:
To create a CMEK-enabled backup in a multi-region instance configuration:
Copy a backup
You can copy a backup of your Spanner database from one instance to another instance in a different region or project. By default, a copied backup uses the same encryption configuration, either Google-managed or customer-managed, as its source backup encryption. You can override this behavior by specifying a different encryption configuration when copying the backup. If you want the copied backup to be encrypted with CMEK when copying across regions, specify the Cloud KMS keys corresponding to the destination regions.
Console
Use the console to copy a backup in a regional instance configuration.
In the Google Cloud console, go to the Instances page.
Click the instance name that contains the database that you want to back up.
Click the database.
In the navigation pane, click Backup/Restore.
In the Backups table, select Actions for your backup and click Copy.
Fill out the form by choosing a destination instance, providing a name, and selecting an expiration date for the backup copy.
Optional: If you want to use a different encryption configuration for your backup, click Show encryption options.
a. Select Cloud KMS key.
a. Select a key from the drop-down list.
The list of keys is limited to the current Google Cloud project. To use a key from a different Google Cloud project, create the database using gcloud CLI instead of the Google Cloud console.
Click Copy.
gcloud
To copy a backup, with a new encryption configuration, to a different instance
in the same project, run the following gcloud spanner backups copy
command:
gcloud spanner backups copy --async \
--source-instance=INSTANCE_ID \
--source-backup=SOURCE_BACKUP_NAME \
--destination-instance=DESTINATION_INSTANCE_ID \
--destination-backup=DESTINATION_BACKUP_NAME \
--expiration-date=EXPIRATION_DATE \
--encryption-type=CUSTOMER_MANAGED_ENCRYPTION \
--kms-keys=KMS_KEY_1[, KMS_KEY_2 ... ]
To copy a backup, with a new encryption configuration, to a different instance
in a different project, run the following gcloud spanner backups copy
command:
gcloud spanner backups copy --async \
--source-backup=SOURCE_BACKUP_NAME \
--destination-backup=DESTINATION_BACKUP_NAME \
--encryption-type=CUSTOMER_MANAGED_ENCRYPTION \
--kms-keys=KMS_KEY_1[, KMS_KEY_2 ... ]
To verify that the copied backup is CMEK encrypted:
gcloud spanner backups describe BACKUP \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID
Client libraries
Node.js
To copy a CMEK-enabled backup in a multi-region instance configuration:
PHP
To copy a CMEK-enabled backup in a multi-region instance configuration:
Ruby
To copy a CMEK-enabled backup in a multi-region instance configuration:
Restore from a backup
You can restore a backup of a Spanner database into a new database. By default, databases that are restored from a backup use the same encryption configuration as the backup itself, but you can override this behavior by specifying a different encryption configuration for the restored database. If the backup is protected by CMEK, the key version that was used to create the backup must be available so that it can be decrypted.
Console
Use the console to restore a backup in a regional instance configuration.
In the Google Cloud console, go to the Instances page.
Click the instance containing the database you want to restore.
Click the database.
In the navigation pane, click Backup/Restore.
In the Backups table, select Actions for your backup and click Restore.
Select the instance to restore and name the restored database.
Optional: If you want to use a different encryption configuration with your restored database, click the slider next to Use existing encryption.
a. Select Cloud KMS key.
a. Select a key from the drop-down list.
The list of keys is limited to the current Google Cloud project. To use a key from a different Google Cloud project, create the database using gcloud CLI instead of the Google Cloud console.
Click Restore.
gcloud
To restore a backup, with a new encryption configuration, run the following
gcloud spanner databases restore
command:
gcloud spanner databases restore --async \
--project=SPANNER_PROJECT_ID \
--destination-instance=DESTINATION_INSTANCE_ID \
--destination-database=DESTINATION_DATABASE_ID \
--source-instance=SOURCE_INSTANCE_ID \
--source-backup=SOURCE_BACKUP_NAME
To verify that the restored database is CMEK encrypted:
gcloud spanner databases describe DATABASE \
--project=SPANNER_PROJECT_ID \
--instance=INSTANCE_ID
For more information, see Restore from a backup.
Client libraries
C#
To restore a CMEK-enabled backup in a regional instance configuration:
C++
To restore a CMEK-enabled backup in a regional instance configuration:
Go
To restore a CMEK-enabled backup in a regional instance configuration:
Java
To restore a CMEK-enabled backup in a regional instance configuration:
Node.js
To restore a CMEK-enabled backup in a regional instance configuration:
To restore a CMEK-enabled backup in a multi-region instance configuration:
PHP
To restore a CMEK-enabled backup in a regional instance configuration:
To restore a CMEK-enabled backup in a multi-region instance configuration:
Python
To restore a CMEK-enabled backup in a regional instance configuration:
Ruby
To restore a CMEK-enabled backup in a regional instance configuration:
To restore a CMEK-enabled backup in a multi-region instance configuration:
View audit logs for the Cloud KMS key
Make sure that logging is enabled for the Cloud KMS API in your project.
In the Google Cloud console, go to Logs Explorer.
Limit the log entries to your Cloud KMS key by adding the following lines to the Query builder:
resource.type="cloudkms_cryptokey" resource.labels.location="KMS_KEY_LOCATION" resource.labels.key_ring_id="KMS_KEY_RING_ID" resource.labels.crypto_key_id="KMS_KEY_ID"
Under normal operations, encrypt and decrypt actions are logged with
INFO
severity. These entries are logged as the zones in your Spanner instance poll the Cloud KMS key about every five minutes.If Spanner fails to access the key, the operations are logged as
ERROR
.