Skbn is a tool for copying files and directories between Kubernetes and cloud storage providers. It is named after the 1981 video game Sokoban. Skbn uses an in-memory buffer for the copy process, to avoid excessive memory consumption. Skbn currently supports the following providers:

• AWS S3
• Minio S3
• Azure Blob Storage
• Google Cloud Storage

1. git
2. dep

Download the latest release from the Releases page or use it with a Docker image

mkdir -p $GOPATH/src/github.com/maorfr && cd $_
git clone https://github.com/maorfr/skbn.git && cd skbn
make

skbn cp \
    --src k8s://<namespace>/<podName>/<containerName>/<path> \
    --dst s3://<bucket>/<path>

skbn cp \
    --src s3://<bucket>/<path> \
    --dst k8s://<namespace>/<podName>/<containerName>/<path>

skbn cp \
    --src k8s://<namespace>/<podName>/<containerName>/<path> \
    --dst abs://<account>/<container>/<path>

skbn cp \
    --src abs://<account>/<container>/<path> \
    --dst k8s://<namespace>/<podName>/<containerName>/<path>

skbn cp \
    --src k8s://<namespace>/<podName>/<containerName>/<path> \
    --dst gcs://<bucket>/<path>

skbn cp \
    --src ... \
    --dst ... \
    --parallel <n>

• n is the number of files to be copied in parallel (for full parallelism use 0)

Skbn copies files using an in-memory buffer. To control the buffer size:

skbn cp \
    --src ... \
    --dst ... \
    --buffer-size <f>

• f is the in-memory buffer size (in MB) to use for files copy. This flag should be used with caution when used in conjunction with --parallel
• The default value for buffer-size is 6.75 MB, and was decided based on benchmark

Skbn supports file copy from and to a Minio S3 endpoint. To let skbn know how your minio is configured, you can set the following environment variables:

AWS_ACCESS_KEY_ID=<your username>
AWS_SECRET_ACCESS_KEY=<your password>
AWS_S3_ENDPOINT=http(s)://<host>:<port>
AWS_S3_NO_SSL=true # disables SSL
AWS_S3_FORCE_PATH_STYLE=true # enforce path style bucket access

skbn cp \
    --src s3://<bucket>/<path> \
    --dst abs://<account>/<container>/<path>

skbn cp \
    --src abs://<account>/<container>/<path> \
    --dst s3://<bucket>/<path>

skbn cp \
    --src k8s://<namespace>/<podName>/<containerName>/<path> \
    --dst k8s://<namespace>/<podName>/<containerName>/<path>

skbn cp \
    --src s3://<bucket>/<path> \
    --dst s3://<bucket>/<path>

skbn cp \
    --src abs://<account>/<container>/<path> \
    --dst abs://<account>/<container>/<path>

Skbn tries to get credentials in the following order:

1. if KUBECONFIG environment variable is set - skbn will use the current context from that config file
2. if ~/.kube/config exists - skbn will use the current context from that config file with an out-of-cluster client configuration
3. if ~/.kube/config does not exist - skbn will assume it is working from inside a pod and will use an in-cluster client configuration

Skbn uses the default AWS credentials chain. In addition, the AWS_REGION environment variable should be set (default is eu-central-1).

Skbn uses AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_ACCESS_KEY environment variables for authentication.

Skbn uses Google Application Default Credentials. Basically, it will first look for the GOOGLE_APPLICATION_CREDENTIALS environment variable. If it is not defined, it will look for the default service account, or throw an error if none is configured.

1. In-cluster example
2. Code example