Skip to content

Commit 27b994d

Browse files
cylkdevcylkdev
committed
refactor s3 adapter
1 parent 87b502c commit 27b994d

19 files changed

Lines changed: 2914 additions & 1729 deletions

‎README.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,41 @@ def deps do
1515
end
1616
```
1717

18+
## Configuration
19+
20+
The following configuration is available:
21+
22+
```elixir
23+
if Mix.env() === :test do
24+
config :cloud_cache, CloudCache.Adapters.S3,
25+
auto_start: true,
26+
sandbox_enabled: true
27+
else
28+
config :cloud_cache, CloudCache.Adapters.S3,
29+
auto_start: true,
30+
sandbox_enabled: false,
31+
retries: [
32+
max_attempts: 10,
33+
base_backoff_in_ms: 10,
34+
max_backoff_in_ms: 10_000
35+
],
36+
access_key_id: [
37+
{:awscli, System.get_env("AWS_PROFILE", "default"), 30},
38+
{:awscli, System.get_env("AWS_PROFILE", "cloud_cache"), 30},
39+
{:system, "CLOUD_CACHE_AWS_ACCESS_KEY_ID"},
40+
{:system, "AWS_ACCESS_KEY_ID"},
41+
:instance_role
42+
],
43+
secret_access_key: [
44+
{:awscli, System.get_env("AWS_PROFILE", "default"), 30},
45+
{:awscli, System.get_env("AWS_PROFILE", "cloud_cache"), 30},
46+
{:system, "CLOUD_CACHE_AWS_SECRET_ACCESS_KEY"},
47+
{:system, "AWS_SECRET_ACCESS_KEY"},
48+
:instance_role
49+
]
50+
end
51+
```
52+
1853
## Testing with LocalStack
1954

2055
`LocalStack` provides a fully functional local AWS cloud stack

‎config/config.exs‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import Config
2+
3+
config :cloud_cache,
4+
auto_start: true,
5+
caches: [CloudCache.Adapters.S3]
6+
7+
config :cloud_cache, CloudCache.Adapters.S3,
8+
sandbox_enabled: false,
9+
profile: "local_stack"
10+
11+
if Mix.env() === :test do
12+
config :cloud_cache, CloudCache.Adapters.S3, sandbox_enabled: true
13+
else
14+
config :cloud_cache, CloudCache.Adapters.S3,
15+
access_key_id: [
16+
{:awscli, System.get_env("AWS_PROFILE", "cloud_cache"), 30},
17+
{:awscli, System.get_env("AWS_PROFILE", "default"), 30},
18+
{:system, "CLOUD_CACHE_AWS_ACCESS_KEY_ID"},
19+
{:system, "AWS_ACCESS_KEY_ID"},
20+
:instance_role
21+
],
22+
secret_access_key: [
23+
{:awscli, System.get_env("AWS_PROFILE", "cloud_cache"), 30},
24+
{:awscli, System.get_env("AWS_PROFILE", "default"), 30},
25+
{:system, "CLOUD_CACHE_AWS_SECRET_ACCESS_KEY"},
26+
{:system, "AWS_SECRET_ACCESS_KEY"},
27+
:instance_role
28+
]
29+
end

‎guides/getting_started/Configuring an AWS CLI Profile for LocalStack.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Configuring an AWS CLI Profile for LocalStack
2+
3+
## Overview
4+
5+
LocalStack emulates AWS services locally, allowing you to test without
6+
real AWS credentials. This guide describes how to configure an AWS CLI
7+
profile named `cloud_cache` that connects to `LocalStack`.
8+
9+
## Steps
10+
11+
1. Create the profile
12+
13+
Open your terminal and run the following command to create a profile
14+
named `cloud_cache`:
15+
16+
```bash
17+
aws configure --profile cloud_cache
18+
```
19+
20+
2. Enter placeholder credentials
21+
22+
When prompted, enter dummy values since `LocalStack` does not require real AWS credentials:
23+
24+
```bash
25+
AWS Access Key ID [None]: test
26+
AWS Secret Access Key [None]: test
27+
Default region name [None]: us-west-1
28+
Default output format [None]: json
29+
```
30+
31+
The AWS CLI will then create entries in two configuration files:
32+
33+
- `~/.aws/credentials` - Stores the actual access keys (`aws_access_key_id` and `aws_secret_access_key`)
34+
for each named profile. Each section (e.g. `[cloud_cache]`) contains the authentication values
35+
used by the AWS CLI and SDKs when making requests.
36+
37+
- `~/.aws/config` - Stores general configuration for each profile, such as the default region, output
38+
format, and optional service endpoints. It complements the credentials file by defining how and where
39+
the profile should connect.
40+
41+
The CLI interactively asks for credentials and settings, then automatically writes them to both files:
42+
43+
- The keys go into `~/.aws/credentials`
44+
- The region and output format go into `~/.aws/config`
45+
46+
3. Edit the configuration files
47+
48+
Open the AWS configuration file located at `~/.aws/config` and ensure it
49+
contains the following:
50+
51+
```ini
52+
[profile cloud_cache]
53+
region = us-west-1
54+
output = json
55+
endpoint_url = http://s3.localhost.localstack.cloud:4566
56+
```
57+
58+
4. Open the credentials file located at `~/.aws/credentials` and verify:
59+
60+
```ini
61+
[cloud_cache]
62+
aws_access_key_id = test
63+
aws_secret_access_key = test
64+
```
65+
66+
5. Test the connection
67+
68+
You can verify that the profile connects correctly by listing S3 buckets:
69+
70+
```bash
71+
aws --profile cloud_cache --endpoint-url http://s3.localhost.localstack.cloud:4566 s3 ls
72+
```
73+
74+
If `LocalStack` is running, you should receive a valid (possibly empty)
75+
response instead of a connection error.

‎guides/getting_started/Setting Up and Running LocalStack.md‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Setting Up and Running LocalStack
2+
3+
## Overview
4+
5+
LocalStack provides a fully functional local AWS cloud stack that allows you to develop and test cloud applications offline.
6+
It runs inside a Docker container and emulates a wide range of AWS services, including S3, DynamoDB, Lambda, and others.
7+
This guide describes how to install, configure, and start LocalStack using the LocalStack CLI.
8+
9+
## Prerequisites
10+
11+
Before proceeding, ensure that your system has the following installed:
12+
13+
- Docker (required to run LocalStack containers). For instructions on how to install Docker, see [Docker's official documentation](https://docs.docker.com/get-docker/).
14+
- A working internet connection for the initial image pull
15+
16+
## Installation
17+
18+
### Brew (macOS or Linux with Homebrew)
19+
20+
Install the LocalStack CLI through the official LocalStack Homebrew tap:
21+
22+
```bash
23+
brew install localstack/tap/localstack-cli
24+
```
25+
26+
## Starting LocalStack
27+
28+
Once installed, you can start LocalStack in detached mode by running:
29+
30+
```bash
31+
localstack start -d
32+
```
33+
34+
You should see output similar to the following:
35+
36+
```bash
37+
__ _______ __ __
38+
/ / ____ _________ _/ / ___// /_____ ______/ /__
39+
/ / / __ \/ ___/ __ `/ /\__ \/ __/ __ `/ ___/ //_/
40+
/ /___/ /_/ / /__/ /_/ / /___/ / /_/ /_/ / /__/ ,<
41+
/_____/\____/\___/\__,_/_//____/\__/\__,_/\___/_/|_|
42+
43+
- LocalStack CLI: 4.9.0
44+
- Profile: default
45+
- App: https://app.localstack.cloud
46+
47+
[17:00:15] starting LocalStack in Docker mode localstack.py:512
48+
preparing environment bootstrap.py:1322
49+
configuring container bootstrap.py:1330
50+
starting container bootstrap.py:1340
51+
[17:00:16] detaching bootstrap.py:1344
52+
```
53+
54+
This command downloads and launches the LocalStack Docker image, setting up the emulated AWS services locally.
55+
56+
## Checking Service Status
57+
58+
After LocalStack starts, you can view the status of available services by running:
59+
60+
```bash
61+
localstack status services
62+
```
63+
64+
You should see a table listing all active services:
65+
66+
```bash
67+
┏━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
68+
┃ Service ┃ Status ┃
69+
┡━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
70+
│ acm │ ✔ available │
71+
│ apigateway │ ✔ available │
72+
│ cloudformation │ ✔ available │
73+
│ cloudwatch │ ✔ available │
74+
│ config │ ✔ available │
75+
│ dynamodb │ ✔ available │
76+
│ s3 │ ✔ available │
77+
...
78+
```
79+
80+
## Integration with AWS CLI
81+
82+
Once LocalStack is running, you can use the AWS CLI to interact with the emulated services by specifying the LocalStack endpoint.
83+
84+
For example, to list S3 buckets:
85+
86+
```bash
87+
aws --profile cloud_cache --endpoint-url http://s3.localhost.localstack.cloud:4566 s3 ls
88+
```
89+
90+
This command connects to the LocalStack S3 service instead of AWS.
91+
92+
For details on setting up AWS credentials, see [Configuring an AWS CLI Profile for LocalStack](./Configuring%20an%20AWS%20CLI%20Profile%20for%20LocalStack.md).
93+
94+
## Stopping LocalStack
95+
96+
To stop LocalStack, you can use the following command:
97+
98+
```bash
99+
localstack stop
100+
```

0 commit comments

Comments
 (0)