Evaluation Guide

Aug 2, 2023
Server General Inc

Transfer General™

Introduction

This document is intended to provide information and suggestions for persons and organizations wishing to evaluate the features and capabilities of Transfer General™ to migrate highly sensitive data from one cloud platform (or a data center) to another platform.

The document assumes that the customer is familiar with the deployment and configuration of a Linux server running Ubuntu Pro 18.04 server 64-bit edition (or any other approved version of Linux) and the customer has administrative control of the machine. As such, we will not be offering specific guidance about installing and configuring the base server.

1: Background

Transfer General™

Transfer General™ (TG) is a secure and high-speed data migration solution that can be used to migrate data between cloud platforms and data centers. TG is made up of two components: Transfer General Main (TGM) and Transfer General Remote (TGR). The TGM is deployed next to the source of the data, while the TGR is deployed at the destination. TG encrypts data at the source and decrypts it at the destination, ensuring that the data remains secure throughout the migration process.

TG offers a number of features that make it a valuable tool for data migration, including:

• Data encryption: TG encrypts data at the source using strong cryptographic algorithms. This ensures that the data remains secure throughout the migration process.

• High-speed data transfer: TG utilizes a private connection to transfer encrypted data between the TGM and TGR. This private connection can achieve speeds of up to 10GBps, making it ideal for migrating large volumes of data.

• Versatile Cloud Platform Support: Transfer General™ empowers data owners to migrate data between various cloud platforms, including Amazon, Azure, Google Cloud, and more. This flexibility allows organizations to choose the most suitable destination for their data without compromising on data security or speed.

• Streamlined Deployment and Management: As a virtual appliance, Transfer General™ can be seamlessly deployed and managed in any environment, making the setup process swift and straightforward for technical professionals. This user-friendly approach saves valuable time and resources during the migration process.

• Data Encryption Key Control: A critical aspect of Transfer General™ is that data owners retain full control over their data encryption keys. This level of control ensures data confidentiality and allows organizations to maintain data ownership throughout the migration journey.

Transfer General™ (TG) stands as the optimal solution for secure and high-speed data migration. By offering robust data encryption, data owners can ensure the confidentiality of their sensitive information throughout the migration process. Simultaneously, Transfer General™ enhances data transfer speed through a private connection, providing technical professionals with the tools they need for efficient and secure data migrations across cloud platforms and data centers. Let’s get started with setting up the necessary infrastructure for the evaluation of this use case.

2: Requirements

TGM Installation Prerequisites:

In addition to the TG binary, you will also need the following:

1. A valid license. If you do not have a license you can get a trial license during configuration.
2. The Server General Product Evaluation Guide or the Quick Start Guide
3. A fully functional Ubuntu Pro 18.04 server is deployed next to your data source within the same network.
4. Unrestricted outbound access to the Internet

TGR Installation Prerequisites:

1. A valid license. If you do not have a license you can get a trial license during configuration.
2. A designated TGM. Please note that the TGR can not operate without a TGM.
3. An open port to accept SSH connections from the TGM.
4. Unrestricted outbound access to the Internet.

Additional Equipment

Initial installation can be performed using a local console and keyboard or via a remote shell window. The following equipment is required for local console installation:

• One PC-compatible keyboard

• One PC-compatible monitor (VGA/HDMI/DVI interface)

• A POSIX-compliant data source (it can be NFS/SAMBA server)

• Network connection that is 300MBps or faster. A slower connection will result in slower data transfer speeds.

• A Cloud Storage bucket.

Network Connectivity

Both the TGM and the TGR must be able to connect to our licensing server and key lockers within the Server General network. No special network configuration is needed to access these machines as long as the TGM and the TGR have unrestricted outbound access to the Internet.

Before you start, verify that:

• TCP ports 80 (HTTP) and 443 (HTTPS) are open for outbound connections.

Personnel

The two main roles recognized by the Transfer General framework are that of a Security Officer and a Data Administrator. For evaluation purposes, a single person may successfully occupy all of these roles.

3: Evaluation/Test Configuration

Transfer General Main - TGM

The following should be configured:

• one virtual Ubuntu Pro 18.04 server

• connectivity to the Internet

• one system user (sgadmin) with root access

• one Security Officer

• one Data Administrator

Transfer General Remote - TGR

The following should be configured:

• one virtual Ubuntu Pro 18.04 server

• connectivity to the Internet

**NFS/SAMBA Server **(optional)

• An optional NFS/SAMBA server to act as a data repository for the data to be migrated. This server is optional. You can also ingest files into the TGM via SFTP/SCP.

4: Testing Goals and Suggestions

The following are some of the goals that you can test when evaluating TG’s secure data migration capabilities:

• Data encryption: Verify that the data is encrypted at the source and decrypted at the destination.
• Data transfer speed: Test the data transfer speed between the TGM and TGR.
• Data integrity: Verify that the data is not corrupted during the migration process.
• User management: Test the user management features of TG.
• Configuration management: Test the configuration management features of TG.

5: Deployment Assumptions

1. You have two valid Transfer General 30-day trial licenses.
2. You have two fully functional Ubuntu Pro servers running 18.04 or have access to Google Marketplace. One server is deployed right next to the source of your data behind your firewall and the other one is at the destination network. Please note that Google Cloud customers can simply deploy the Transfer General virtual instance available within Google Marketplace instead of a Ubuntu server.
3. You have the Transfer General binary in your possession [not needed for Google Cloud customers].
4. You have “root” privileges on the above-mentioned servers.

6: Transfer General Threat Model for Data Migration

It is important to understand the threat model that Transfer General uses and how it mitigates those threats.

Actor	Threat	Mitigation by Transfer General
External Attacker	Man-in-the-Middle (MITM) Attack	- Transfer General encrypts data at the source ensuring that data remains encrypted throughout the migration journey.
		- Strong cryptographic algorithms and key management practices are employed to protect against unauthorized decryption.
		- Transfer General utilizes private, point-to-point connections to minimize the risk of interception and unauthorized access to the data being migrated.
		- Encryption keys are generated and managed securely by Transfer General, and the customer retains full control over these keys to prevent unauthorized access.
Insider	Unauthorized Data Access or Disclosure	- Transfer General enforces strict role-based access controls to ensure that only authorized personnel can access and manage the migration process.
		- Detailed logging and monitoring mechanisms are in place to track and detect any unauthorized access attempts or suspicious activities.
		- Transfer General employs strong access management practices to limit the scope of access to only the data required for the migration, reducing insider threat risks.
Data Loss	Data Corruption or Loss during Migration	- Transfer General implements data integrity checks at both the source and destination to ensure data consistency during the migration process.
		- Robust error detection and correction mechanisms are utilized to handle any data transmission errors that may occur during the migration.
		- Transfer General conducts comprehensive testing and validation before, during, and after the migration process to ensure data integrity and minimize data loss risks.
Layer-2/3 Provider	Data interception and tampering in transit	N/A (Data is already encrypted at the source and remains encrypted during transit)
	Unauthorized access to data at rest in TGM or TGR	Encryption at rest on both source (TGM) and destination (TGR)

7: Transfer General Main (TGM) Deployment

Let’s assume that you are looking to migrate your highly sensitive data from AWS (Amazon Web Service) to Google Cloud.

Google Cloud customers can skip this step

Assuming that you already have an Ubuntu Pro 18.04 server installed right next to the source of your data in AWS:

$ ssh -l<user> <IP Address of your server>

# sudo -i

# mkdir tg

Copy your tg binary into this directory:

# cd tg

The next command will install everything that is needed to run Transfer General. It will take some time to finish this installation:

# ./tg-install

Once everything is installed you’ll then proceed to configure the machine using our web interface.

8: Configuring TGM

Step 1: Accessing the TGM Configuration Interface

Open a web browser on your local machine. In the address bar, enter the following URL, replacing <IP address of the machine> with the actual IP address of your TGM server: https://<IP address of the machine>

A warning about the SSL certificate might appear. Please click on the "Accept" option to proceed and initiate the installation process.

Please click on “Advanced” and continue the process.

Step 2: Accepting the License Agreement

After accessing the TGM configuration interface, carefully review the presented license agreement.

Step 3: Entering License Information

1. Following the license agreement acceptance, you'll be prompted to enter license information.
2. If you possess a valid Transfer General license, input the license string as required.
3. For those seeking a 30-day trial license, provide a valid email address. A verification code will be sent to your email.
4. Enter the received security code and your company name. Proceed with these details to initiate the license request process.

Step 4: Selecting License Type

1. Upon successful completion of the license request, you will be asked to specify the license type you desire.
2. The options are Transfer General Main (TGM) or Transfer General Remote (TGR).
3. Choose "Transfer General Main" (TGM) as the license type.
4. Click on "Get Trial License" to submit your selection.
5. Your request will be processed, and upon approval, your TGM machine will be licensed accordingly. Note that the selected license type cannot be changed after submission.

Understanding Transfer General Administrators

In the Transfer General framework, role-based access control (RBAC) forms the cornerstone of data security and management. RBAC is a robust system that partitions system and data management responsibilities among different roles, ensuring a tightly controlled circle of trust around application data. This strategic division of administrative tasks guarantees that each administrator is granted access and control only over the specific functions necessary for fulfilling their designated responsibilities.

The primary roles within the RBAC framework of the Transfer General are:

Security Officer (SO): The Security Officer holds a position of utmost trust within the organization. Their role involves overseeing the administration of other administrators, specifically Data Administrators. The Security Officer is empowered to initiate, reassign, or revoke Data Administrators. Crucially, the Security Officer manages the initiation of new Data Administrators in case of forgotten Data Master Keys (DMKs). It's important to note that direct access to the protected data sets is not granted to the Security Officer. During initial configuration, the Security Officer establishes their unique "Security Officer's Master Key" (SMK), which is critical for authentication purposes.

Data Administrator (DA): Data Administrators are responsible for the day-to-day management of assigned data sets. They have the authority to create, enforce, and remove security policies for their respective data sets. A Data Administrator chooses their Data Master Key (DMK), a passphrase required for executing operations. While Data Administrators are integral to the Transfer General trust model and possess the ability to view data in clear text, they do not have direct access to the RBAC capabilities that govern RBAC. These capabilities are abstracted from the user interface.

"sgadmin" User: The "sgadmin" user holds a pivotal role as a system user with sudo privileges. Operating within the Transfer General trust model, the "sgadmin" user is granted access to specific functions critical for system management. This includes configuration and maintenance tasks necessary for the smooth operation of Transfer General. A password is assigned to the "sgadmin" user for accessing the web management console.

It's important to emphasize that the Transfer General user interface deliberately conceals the complexities of RBAC capabilities, ensuring a user-friendly experience without compromising security. While the specific actions associated with RBAC roles are not directly exposed, they play a crucial role in safeguarding data assets and enabling seamless data migration within the Transfer General ecosystem.

Understanding the user roles within Transfer General is essential, as they fall into distinct categories:

User	SG Admin (sgadmin)	Security Officer	Data Administrator
Linux OS user	Yes	No	No
Role	A special user. All SG commands are executed in the context of this user.	Overall in charge. Manages other administrators.	Trusted with the responsibility of managing one of the more sensitive data sets.
Authentication	Password or key pair	Master key (SMK)	Master key (DMK)

Step 6: Configuring **the “sgadmin” user:

Configuration Details:

1. The "sgadmin" user is a system user granted sudo privileges.
2. This user resides within the trust model, signifying implicit trust.
3. A password must be assigned to the "sgadmin" user.
4. This password is vital for accessing the web management console in the future.
5. Note that the password must be alpha-numeric and contain a minimum of 10 characters.

Step 7: Configuring Security Officer Master Key

The Security Officer (SO) is crucial in managing Data Administrators and overseeing key security aspects within Transfer General:

1. There is a single SO for each Transfer General installation.
2. The SO assumes responsibility for generating and controlling its unique data master key, known as the SMK.
3. The SMK serves for SO authentication and certain cryptographic operations.
4. The SO holds the authority to add or revoke a Data Administrator (DA).
5. The SO can reassign a DA’s management duties to another DA.
6. Notably, the SO is not permitted to access protected data sets in cleartext.

Configuration Steps:

1. Assign an SMK to your Security Officer.
2. The SMK should be an alphanumeric string of at least 16 characters in length.

Important Considerations:

• Safeguard the SMK diligently. Losing the SMK may result in irrecoverable data loss. No system backdoors are available for data recovery.
• It is advised to write down the SMK on a secure physical medium, such as a piece of paper, and store it in a safe location.

Step 8: Configuring Data Administrator’s Master Key

The Data Administrator (DA) assumes a trusted role in managing sensitive data sets within the Transfer General framework. This role involves defining security policies that govern data sets under their purview. These policies encompass critical security parameters, such as encrypted directories, encryption algorithms, and authorized system users for accessing protected directories. Notably, Transfer General streamlines this process by providing pre-configured security policies. The default data admin user-id is "DataAdmin1".

Steps for Configuration:

1. Proceed to assign a Data Master Key (DMK) to your Data Administrator.
2. The DMK is a confidential and exclusive key, known only to you. Transfer General does not have access to this key.
3. The DMK is instrumental in encrypting the data encryption key, affording you the sole capability to reconstruct the data encryption key using your DMK.

Note: Upon configuring the DMK, Transfer General initiates a series of operations, and this phase may entail a duration before transitioning to the subsequent interface, referred to as the "Dashboard".

Step 9: Configuring Data Source

To populate the Transfer General Main (TGM) with your data, various methods are available. Choose the one that aligns with your system configuration and requirements. Be mindful not to remove your source data following the data transfer to the TGM.

Option 1: SCP or SFTP

For Microsoft Windows Users (using an SCP or SFTP client):

1. Download a suitable SCP or SFTP client that supports UTF-8 filenames, such as WinSCP.
2. Using the SCP tool, establish a connection to the TGM using these settings:
   • File protocol: SFTP
   • Hostname: TGM's IP address
   • Port number: 22
   • Username: "uploader"
   • Password: Accessible from your Dashboard.
3. The destination directory on the TGM must be: /home/uploader/upload.
4. Ensure the Data Vault is open before initiating the data transfer via SFTP or WinSCP.

For Linux Users (using SCP):

1. Execute the following command:

$ scp PATH_TO_FILES USERNAME@IP_ADDRESS:/home/uploader/upload

Replace placeholders:

• PATH_TO_FILES: Path to the files for copying.
• USERNAME: "uploader"
• IP_ADDRESS: TGM's IP address.

2. When prompted, provide the password, accessible from your Dashboard.
3. Ensure the Data Vault is open before initiating the data transfer via SCP.

Option 2: Use SMB to Transfer Data

To activate SMB file sharing, follow these steps:

1. Input the SMB share path in this format:

\\\<IP address of your Windows server>\<exported share>`

2. Identify the exported share on your Windows server beforehand.

Option 3: Use NFS Share to Copy Data

For Linux Users:

1. Specify the IP address of your NFS server.
2. Find which share is exported by using the command:

# cat /etc/exports

3. Enter the NFS share path

Select the approach that best suits your environment and data migration needs to facilitate seamless data transfer to the Transfer General Main (TGM).

NOTE: YOU MUST MAKE SURE THE DATA VAULT IS OPEN BEFORE COPYING THE DATA OVER TO THE TGM USING SFTP OR WINSCP.

Step 10: Ingesting Data

After configuring your data source, the subsequent step involves data ingestion into the Transfer General Main (TGM).

1. Click the "Load Data" Button: Initiate the data ingestion process by clicking the "Load Data" button. This action ensures the data vault's accessibility, enabling the TGM to encrypt the data during ingestion.
2. Encryption and Data Security: It is crucial to emphasize that data ingested into the TGM is never stored on the disk in an unencrypted format. Instead, the data encryption key is employed to encrypt the data, ensuring its security. You, as the authorized user, possess the capability to reconstitute the encryption key through your passphrase (DMK).
3. Manual Data Copy: If you opt to manually copy files into the /home/uploader/upload directory, it is imperative to ensure the data vault is open. The "Open Vault" command is accessible via your dashboard, and using it guarantees that the data transfer occurs smoothly.
4. Processing Time: Depending on the volume of data being ingested, the process may take some time to complete. It is advisable to communicate your requirements to your Server General sales representative. This step facilitates appropriate guidance and ensures the possibility of deploying a larger TGM server, if necessary. Such an upgrade may involve increasing CPU and memory resources.
5. Network Security: To preserve data security and expedite the transfer process, it is paramount that source data is transferred into the TGM via a local area network (LAN).

Note: Before manually copying data to the TGM (/home/uploader/upload directory), ensure that the data vault is open to prevent any disruptions or issues in the data ingestion process.

9: Adding TGR

In our pursuit of migrating highly sensitive data from AWS to Google Cloud, the subsequent phase involves the incorporation of a Transfer General Remote (TGR) instance within the Google Cloud environment. The TGR solution can be seamlessly obtained via Google Marketplace, eliminating the need for a separate Ubuntu server installation.

Procedure:

1. Access Google Marketplace: Within the Google Cloud environment, navigate to Google Marketplace and search for "Transfer General." Locate the Transfer General application and initiate the configuration process.
2. TGR Configuration: Follow the steps similar to those employed during the TGM installation. However, in this instance, choose the machine type as "Transfer General Remote (TGR)." This will ensure the appropriate setup for the TGR within the Google Cloud infrastructure.
3. Public SSH Key: To establish secure communication between the TGM and the newly added TGR, you will need to input the public SSH key of the TGM's "sgadmin" user. This key is conveniently accessible within the "Add TGR" section of your TGM dashboard:

1. Copy SSH Key: Retrieve the public SSH key associated with the TGM's "sgadmin" user from the TGM dashboard.



2. Paste into TGR Configuration: Within the TGR setup process, paste the copied SSH key into the designated field. This action ensures a secure communication link between the TGM and TGR instances.

4. Proceed and Configuration: After pasting the SSH key, click "Proceed" to advance to the next stage of the configuration process.
5. TGM Administrator Configuration: At this juncture, the TGM administrator assumes the responsibility of finalizing the TGR configuration. This entails providing the necessary information to complete the "Add TGR" step.

6. User Identification and DMK: Enter the user ID, which is "DataAdmin1," and the Data Master Key (DMK) generated during the initial configuration. The DMK serves a crucial role in data encryption and security.
7. Verify Configuration: Return to the TGR environment and initiate the "Check Configuration" process. This action confirms the successful integration and configuration of the TGR instance.
8. TGR Dashboard: Upon successful completion of the configuration verification, access the TGR dashboard. This interface provides a comprehensive overview of the TGR's functionalities and settings.

Only the Data Administrator possesses the authority to add a TGR instance. As such, the TGR setup requires the entry of specific user credentials and keys to ensure a seamless and secure data migration process.

By effectively incorporating a Transfer General Remote (TGR) instance within the Google Cloud infrastructure, you are establishing a pivotal component for securely managing and migrating your highly sensitive data.

10: Direct Data Push

In the culminating phase of the data migration process, you will initiate a direct push of data from the Transfer General Main (TGM) instance hosted on AWS to the Transfer General Remote (TGR) instance situated within the Google Cloud environment. This direct push capability is readily accessible through the TGM dashboard, serving as the conduit for secure and efficient data transfer.

Guidelines:

1. Access TGM Dashboard: Navigate to the TGM dashboard, which provides a comprehensive control center for managing the data migration process.
2. Initiate Direct Push: Locate and activate the "Direct Push" feature within the TGM dashboard. This functionality is specifically designed to facilitate the seamless transfer of data between the TGM and TGR instances.
3. Encrypted Data Transfer: Upon activating the direct push, the TGM will embark on transferring encrypted data from its own data vault to the corresponding data vault within the TGR instance. This encrypted data transmission ensures the utmost data security during the migration journey.

It is essential to underscore that the direct push mechanism serves as the conduit for securely transmitting encrypted data from the TGM on AWS to the TGR hosted on Google Cloud. The comprehensive report generated during this phase offers a detailed overview of the data transfer process, further enhancing your ability to monitor and manage the migration endeavor.

By executing the direct data push, you are achieving the final step in migrating highly sensitive data across cloud platforms, culminating in the successful and secure transfer of your valuable information to its designated destination within the Google Cloud environment.

You’ll see a report that will look like this:

11: Copying Data from TGR's Data Vault

In this phase, as the administrator of the Transfer General Main (TGM) instance, you will wield control over the data vault situated within the Transfer General Remote (TGR) instance. This authority empowers you to access and manage the data stored within the TGR's data vault. Specifically, you will have the capability to initiate the extraction of data from the TGR's data vault and copy it onto your designated file server using the secure file transfer protocol (SFTP).

Procedure:

1. Open Data Vault: Commence by accessing the TGM dashboard, your command center for overseeing the entire data migration process.
2. Access TGR Data Vault: Within the TGM dashboard, navigate to the section that allows you to manage the TGR instance. As the TGM administrator, you possess the privilege to control and manipulate the data vault within the TGR.
3. Unlock Data Vault: Initiate the process of unlocking the data vault residing within the TGR instance. This action is pivotal, as it grants you the authorization to access the encrypted data stored within the vault.
4. Utilize SFTP for Data Transfer: Once the data vault is unlocked, leverage the secure file transfer protocol (SFTP) to establish a secure and encrypted connection between the TGR instance and your designated file server.
5. Copy Data from TGR: Utilizing the established SFTP connection, navigate to the TGR's data vault and commence the process of copying the desired data onto your file server. This methodical data transfer ensures that the data's integrity and security are upheld throughout the process.
6. Confirm: While the data is being copied, vigilantly monitor the process for any indications of completion or potential issues. Once the data transfer is finalized, confirm the successful transfer of data from the TGR's data vault to your designated file server.

It is imperative to underscore that during this phase, the TGM administrator assumes the role of orchestrating the extraction of data from the TGR's data vault. Through the utilization of SFTP, you possess the capability to securely copy the encrypted data onto your own file server. This meticulous and secure data transfer process guarantees the confidentiality and integrity of the migrated data.

By competently executing this step, you successfully conclude the data migration journey, effectively transferring highly sensitive data from the TGR instance within the Google Cloud environment to your designated file server, thus culminating in a seamless and secure migration endeavor.

Note: If you need to evaluate our high-speed data migration feature, then please contact your sales representative and they’ll work with you to ensure a proper setup is put in place.