Enable or disable online transfer

The Transfer Appliance has two modes:

  • Offline transfer: Data is copied to the appliance until it is full. The appliance is shipped back to Google and the copied data is moved to your Cloud Storage bucket.

  • Online transfer: Data copied to the appliance is streamed to your Cloud Storage bucket. After the data is uploaded to your Cloud Storage bucket, it is removed from the appliance. Online transfer compresses and encrypts the data in transit and accelerates data transfer compared to the gcloud CLI and other command-line tools.

You can toggle between offline and online mode.

Download Appliance Service Account Key

Once the project setup is completed, prepare a service account key:

  1. Create a service account key for the appliance service account. This is a one time download key.

  2. Rename the key to key.json, and save it to a safe location.

  3. Once you've received your appliance, upload key.json to the /tmp directory on the appliance.

Set to online mode

Before enabling online transfer mode for the first time, follow the instructions to complete project setup.

Outbound connectivity is required in order to enable and use online mode. To enable online transfer mode, follow the instructions below:

  1. Verify your outbound connectivity from the appliance to the HTTPS endpoints listed below:

    • https://gcr.io

    • https://storage.googleapis.com

    • https://oauth2.googleapis.com

    • https://storagetransfer.googleapis.com

    • https://pubsub.googleapis.com

    Run the following command using each of the HTTPS endpoints:

    ping URL

    If the command fails, it may indicate that DNS is misconfigured on the appliance. If the appliance was configured to use a static IP address, update it to use DHCP:

    ta config --data_port=PORT --ip=dhcp

    Replace PORT with the data port you are using on the appliance, either RJ45 or QSFP.

  2. Copy the service key downloaded in Download Appliance Service Account Key to the appliance's tmp folder, by doing the following:

    Windows

    1. Using the SCP tool, connect to the appliance using the following settings:

      • File protocol: SFTP
      • Host name: The IP address for the appliance.
      • Port number: 22
      • Username: The appliance username provided by the Transfer Appliance Team.
      • Password: The appliance password provided by the Transfer Appliance Team.

    2. Copy the file to the /tmp folder

    Linux

    1. Run the following command:

      scp PATH_TO_KEY_FILE/key.json USERNAME@IP_ADDRESS:/tmp

      Replace the following:

      • PATH_TO_KEY_FILE: The path to the key file that you are copying.
      • USERNAME: The appliance username provided by the Transfer Appliance Team.
      • IP_ADDRESS: The IP address for the appliance.

    2. When prompted, enter the appliance password provided by the Transfer Appliance Team.

    Apple macOS

    1. Run the following command:

      scp PATH_TO_KEY_FILE/key.json USERNAME@IP_ADDRESS:/tmp

      Replace the following:

      • PATH_TO_KEY_FILE: The path to the key file that you are copying.
      • USERNAME: The appliance username provided by the Transfer Appliance Team.
      • IP_ADDRESS: The IP address for the appliance.

    2. When prompted, enter the appliance password provided by the Transfer Appliance Team.

  3. To set the appliance to online mode, run the following command:

    ta online

    The following sample response resembles the output that is returned:

    
Starting containers for online transfer...
Copied new credentials.          ## Or, Using old credentials
Running binary to enable online transfer.
This may take several minutes to finish...

Containers have been successfully started. You can transfer data in
online mode.

  4. Optional: change the transfer destination bucket and the job object prefix.

    ta online --bucket=BUCKET --object_prefix=OBJECT_PREFIX

    Replace BUCKET with the destination bucket and OBJECT_PREFIX with the object prefix you are updating to.

    If the command is successful, it returns output resembling the output from the step where you run the online transfer without these two arguments.

To troubleshoot online mode, see Troubleshooting online mode.

Set to offline mode

To specify offline mode, run the following command:

ta offline

The following sample response resembles the output that is returned:


Putting the appliance in offline mode...
Appliance is successfully put into the offline mode.