Agent-based transfer permissions

Before creating an agent-based transfer, you must configure permissions for the following entities:

• The user or user-managed service account being used to create the transfer. This is the account that is signed in to the Google Cloud console, or the account that is specified when authenticating to the gcloud CLI. The user account can be a regular user account, or a user-managed service account.

• The Google-managed service account, also known as the service agent, used by Storage Transfer Service. This account is generally identified by its email address, which uses the format project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com.

• The transfer agent account that provides Google Cloud permissions for transfer agents. Transfer agent accounts use the credentials of the user installing them, or the credentials of a user-managed service account, to authenticate.

For more information on granting IAM roles, see Granting, changing, and revoking access to resources.

Simplest method of granting permissions

The gcloud CLI can be used to grant the required permissions to the user / user-managed service account and to the Google-managed service account. These permissions allow the user to create, edit, and delete transfer jobs, and set or modify bandwidth limits.

If these permissions are too broad for your organization's policies, refer to the later sections in this document for the minimum permissions required by Storage Transfer Service.

To inspect existing permissions and print out any missing roles, run the following command:

gcloud transfer authorize

To automatically apply those roles, use the --add-missing flag:

gcloud transfer authorize --add-missing

To grant permissions to a user-managed service account, pass the service account key file:

gcloud transfer authorize --add-missing --creds-file=path/to/key.json

For instructions on creating a service account, see Creating and managing service accounts.

The command grants the following permissions.

• To the user / user-managed service account:

  • roles/owner
  • roles/storagetransfer.admin
  • roles/storagetransfer.transferAgent
  • roles/storage.objectAdmin
  • roles/pubsub.editor

• To the Google-managed service account:

  • roles/storage.admin
  • roles/storagetransfer.serviceAgent

For instructions on installing the gcloud CLI, see the gcloud quickstart.

User or user-managed service account permissions

This section covers the roles required for the accounts managing and executing transfers. Your organization's requirements will dictate the exact roles for each persona; this section assumes that you will create an administrator and a user.

Administrator accounts

Storage Transfer Service administrator accounts manage transfer agents, set bandwidth usage limits, and delete transfer jobs.

To set up an administrator account, assign the following IAM permissions and roles:

User accounts

Storage Transfer Service user accounts are used to create and execute transfers. These accounts typically don't have access to delete transfer jobs.

A user account can be a Google Cloud console user, or a service account. If you're using a service account, the method you use to pass the credentials to Storage Transfer Service varies depending on the interface you use.

To set up a user account, assign the following permissions and roles to the account:

Google-managed service account permissions

Storage Transfer Service uses a Google-managed service account to move your data. This service account is automatically created the first time you create a transfer job, call googleServiceAccounts.get, or visit the job creation page in the Google Cloud console.

The service account's format is typically project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com. To retrieve your service account identifier, use the googleServiceAccounts.get API call.

Automatically assigning roles

There are two ways to automatically assign the correct roles to the Google-managed service account:

• Create your transfer using the Google Cloud console. The console automatically applies the required permissions.

• Use gcloud transfer authorize --add-missing. See Simplest method of granting permissions.

Manually assigning roles

To allow the Google-managed service account access to resources needed to complete transfers, assign the following roles, or equivalent permissions, to the service account.

See Grant the required permissions for instructions.

Transfer agent account permissions

Storage Transfer Service transfer agents can be run with either the user's account, or with a service account.

To set up a transfer agent service account or user account running the transfer agents, assign the following role:

Source and destination permissions

You must also ensure that the agent account has the correct permissions to access the source data and to write to the destination.

File system, S3-compatible storage, or HDFS to Cloud Storage

If your transfer destination is a Cloud Storage bucket, the transfer agent needs the following permissions on the destination bucket. See Add a principal to a bucket-level policy for instructions.

To grant these permissions, grant the following role:

• Storage Object Admin (roles/storage.objectAdmin)

Or, create a custom role with the specific permissions and grant the custom role.

Additional permissions are required to enable multipart uploads.

Cloud Storage to file system

If your transfer's source is a Cloud Storage bucket, the transfer agent needs the following permission on the source bucket.

To grant this permission, grant the following role:

• Storage Object Viewer (roles/storage.objectViewer)

Or, create a custom role with the single permission and grant the custom role.

File system to file system

If your transfer is between two file systems, the transfer agent needs the following permissions on the intermediate bucket.

To grant these permissions, grant the following role:

• Storage Object Admin (roles/storage.objectAdmin)

Or, create a custom role with the specific permissions and grant the custom role.

Additional permissions are required to enable multipart uploads.

Multipart uploads

To enable multipart uploads for file system to Cloud Storage transfers or transfers between file systems, additionally grant the following permissions to the agent.

• For transfers to Cloud Storage, assign the permissions to the destination bucket.
• For transfers between file systems, assign the permissions to the intermediate bucket.

To grant these permissions, grant the following role:

• Storage Object Admin (roles/storage.objectAdmin)

Or, create a custom role with the specific permissions and grant the custom role.