The Git LFS (Large File Storage) S3 Proxy is a server application that acts as a proxy between a Git LFS client and an Amazon S3 bucket. It allows seamless integration of Git LFS with S3 for storing and retrieving large files. This proxy handles Git LFS batch requests, generating pre-signed URLs for S3 upload and download operations.

Before using the Git LFS S3 Proxy, ensure that the following dependencies are installed:

• Node.js
• Yarn (Package Manager for Node.js)

Install necessary Node.js packages using Yarn by running:

yarn install

Set up AWS credentials either through environment variables or a profile in the AWS CLI. The proxy uses the AWS SDK for JavaScript v3 to interact with S3.

Environment variables:

• AWS_REGION: AWS region (default is "eu-west-1")
• AWS_PROFILE: AWS CLI profile name

Or:

• AWS_REGION: AWS region (default is "eu-west-1")
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN

Configure the proxy by setting the following environment variables:

• S3_BUCKET: S3 bucket name where Git LFS objects will be stored.

Start the Git LFS S3 Proxy by executing the following command:

yarn start

Or using the prebuild docker image:

docker run \
  --name git_lfs_proxy \
  -v ~/.aws:/root/.aws \
  -e AWS_PROFILE=my-aws-profile \
  -e S3_BUCKET=my-git-lfs-s3-bucket \
  -p 3000:3000 \
  msangals/git-lfs-s3-proxy

The proxy will be accessible at http://localhost:3000 by default.

• POST /:organizationName/:repositoryName/objects/batch

Handles Git LFS batch requests, providing pre-signed URLs for S3 upload and download operations.

• Accepts Git LFS batch requests in JSON format.

• Responds with Git LFS batch response containing pre-signed URLs.

• GET /list-objects

Lists objects in the configured S3 bucket.

• Responds with a JSON object containing a list of object keys.

• DELETE /delete-all-objects

Deletes all objects in the configured S3 bucket.

• Responds with a success message if deletion is successful.

• GET /health

Provides a health check endpoint to verify the status of the proxy.

• Responds with a JSON object indicating the status.

In case of errors during batch request processing or S3 operations, appropriate HTTP status codes are returned, and error details are logged to the console.

• Ensure that AWS credentials have the necessary permissions for S3 operations.
• Configure proper firewall rules and access controls for the server running the proxy.

The Git LFS S3 Proxy facilitates the integration of Git LFS with Amazon S3, providing a scalable solution for managing large files in Git repositories.

To ensure that Git LFS operations are directed through the Git LFS S3 Proxy, you need to configure the Git LFS URL in your repository. This is achieved by creating a .lfsconfig file in the root of your Git repository.

1. Create a file named .lfsconfig in the root directory of your Git repository.

2. Open the .lfsconfig file in a text editor and add the following content:

   iniCopy code

   [lfs] url = http://localhost:3000/:organizationName/:repositoryName/objects/batch

   Replace :organizationName and :repositoryName with the appropriate values for your organization and repository.

   Example .lfsconfig file content:

   iniCopy code

   [lfs]
       url = http://localhost:3000/myorganization/myrepository/objects/batch
   

3. Save and commit the .lfsconfig file to your Git repository.

The .lfsconfig file specifies the URL that Git LFS should use for batch operations such as uploading and downloading large files. By setting this URL to your Git LFS S3 Proxy endpoint, you ensure that all Git LFS traffic is directed through the proxy, allowing it to generate pre-signed URLs for S3 operations.

Make sure to replace the placeholders in the URL with the actual organization and repository names. This URL format corresponds to the Git LFS batch endpoint provided by the Git LFS S3 Proxy.

If you are adding Git LFS to an existing repository, ensure that all contributors update their local repositories with the new .lfsconfig file. This can be done by pulling the latest changes from the remote repository after the .lfsconfig file has been committed.

With the .lfsconfig file in place, your Git LFS client will use the specified proxy URL for all interactions with large files, seamlessly integrating the Git LFS S3 Proxy into your workflow.