You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: website/docs/docs/quickstarts/dbt-cloud/starburst-galaxy-qs.md
+40-26Lines changed: 40 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,9 +5,9 @@ id: "starburst-galaxy"
5
5
sidebar_label: "Starburst Galaxy quickstart"
6
6
---
7
7
8
-
In this quickstart guide, you'll learn how to use dbt Cloud with Starburst Galaxy. It will show you how to:
8
+
In this quickstart guide, you'll learn how to use dbt Cloud with [Starburst Galaxy](https://www.starburst.io/). It will show you how to:
9
9
10
-
- Load data to the Amazon S3 bucket. This guide uses AWS as the cloud service provider for demonstrative purposes. Starburst Galaxy also supports other providers such as Google Cloud, Microsoft Azure, and more.
10
+
- Load data to the Amazon S3 bucket. This guide uses AWS as the cloud service provider for demonstrative purposes. Starburst Galaxy also [supports other providers](https://www.starburst.io/platform/connectors/) such as Google Cloud, Microsoft Azure, and more.
11
11
- Connect Starburst Galaxy to the Amazon S3 bucket.
12
12
- Create tables with Starburst Galaxy.
13
13
- Connect dbt Cloud to Starburst Galaxy.
@@ -23,22 +23,24 @@ You can check out [dbt Fundamentals](https://courses.getdbt.com/courses/fundamen
23
23
24
24
## Prerequisites
25
25
- You have a [dbt Cloud account](https://www.getdbt.com/signup/).
26
-
- You have [a trial Starburst Galaxy account](https://www.starburst.io/platform/starburst-galaxy/.). For details, refer to the [getting started guide](https://docs.starburst.io/starburst-galaxy/get-started.html) in the Starburst Galaxy docs.
27
-
- You have an AWS account with permissions to upload data to an S3 bucket. This guide uses AWS as the cloud service provider for demonstrative purposes.
28
-
- For AWS (S3) authentication, you will need either an AWS access key and AWS secret key with access to the bucket, or you will need a cross account IAM role with access to the bucket. For details, refer to these Starburst Galaxy docs:
26
+
- You have [a trial Starburst Galaxy account](https://www.starburst.io/platform/starburst-galaxy/). For details, refer to the [getting started guide](https://docs.starburst.io/starburst-galaxy/get-started.html) in the Starburst Galaxy docs.
27
+
- You have an AWS account with permissions to upload data to an S3 bucket.
28
+
- For Amazon S3 authentication, you will need either an AWS access key and AWS secret key with access to the bucket, or you will need a cross account IAM role with access to the bucket. For details, refer to these Starburst Galaxy docs:
29
29
-[AWS access and secret key instructions](https://docs.starburst.io/starburst-galaxy/security/external-aws.html#aws-access-and-secret-key)
30
30
-[Cross account IAM role](https://docs.starburst.io/starburst-galaxy/security/external-aws.html#role)
31
31
32
32
## Load data to an Amazon S3 bucket {#load-data-to-s3}
33
33
34
-
To be able to use Starburst Galaxy to create tables that can be transformed with dbt Cloud, you will need to upload the sample data to the object storage service for your cloud provider.
34
+
Using Starburst Galaxy, you can create tables and also transform them with dbt. Start by loading the Jaffle Shop data (provided by dbt Labs) to your Amazon S3 bucket. Jaffle Shop is a fictional cafe selling food and beverages in several US cities.
35
35
36
-
1. Download these CSV files (Jaffle Shop sample data) to your local machine:
36
+
1. Download these CSV files to your local machine:
2. Upload these files to S3. When uploading these files, you must create the following folder structure and upload the appropriate file to each folder:
41
+
2. Upload these files to S3. For details, refer to [Upload objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html) in the Amazon S3 docs.
42
+
43
+
When uploading these files, you must create the following folder structure and upload the appropriate file to each folder:
42
44
43
45
```
44
46
<bucket/blob>
@@ -52,7 +54,9 @@ To be able to use Starburst Galaxy to create tables that can be transformed with
52
54
```
53
55
54
56
## Connect Starburst Galaxy to the Amazon S3 bucket
55
-
If your Starburst Galaxy instance is not already connected to your S3 bucket, you need to create a cluster, create a catalog that connects to the S3 bucket, associate the new catalog (your data source) to your new cluster, and configure access control settings.
57
+
If your Starburst Galaxy instance is not already connected to your S3 bucket, you need to create a cluster, create a catalog that connects to the S3 bucket, associate the new catalog (your data source) to your new cluster, and configure privilege settings.
58
+
59
+
In addition to Amazon S3, Starburst Galaxy supports many other data sources. To learn more about them, you can refer to the [Catalogs overview](https://docs.starburst.io/starburst-galaxy/catalogs/index.html) in the Starburst Galaxy docs.
56
60
57
61
1. Create a cluster. Click **Clusters** on the left sidebar of the Starburst Galaxy UI, then click **Create cluster** in the main body of the page.
58
62
2. In the **Create a new cluster** modal, you only need to set the following options. You can use the defaults for the other options.
@@ -62,49 +66,52 @@ If your Starburst Galaxy instance is not already connected to your S3 bucket, yo
62
66
When done, click **Create cluster**.
63
67
64
68
1. Create a catalog. Click **Catalogs** on the left sidebar of the Starburst Galaxy UI, then click **Create catalog** in the main body of the page.
65
-
2. On the **Create a data source** page, select Amazon S3.
66
-
3. In the **Name and description** section, fill out the fields.
67
-
4. In the **Authentication to S3** section, select the [mechanism you chose to connect to the bucket](#prerequisites).
69
+
2. On the **Create a data source** page, select the Amazon S3 tile.
70
+
3. In the **Name and description** section of the **Amazon S3** page, fill out the fields.
71
+
4. In the **Authentication to S3** section of the **Amazon S3** page, select the [AWS (S3) authentication mechanism](#prerequisites) you chose to connect with.
68
72
5. In the **Metastore configuration** section, set these options:
69
73
- **Default S3 bucket name** — Enter the name of your S3 bucket you want to access.
70
74
- **Default directory name** — Enter the folder name of where the Jaffle Shop data lives in the S3 bucket. This is the same folder name you used in [Load data to an Amazon S3 bucket](#load-data-to-s3).
71
75
- **Allow creating external tables** — Enable this option.
72
76
- **Allow writing to external tables** — Enable this option.
73
77
6. In the **Default table format** section, choose **Hive**.
74
78
79
+
The **Amazon S3** page should look similar to this, except for the **Authentication to S3** section which is dependant on your setup:
80
+
75
81
<Lightbox src="/img/quickstarts/dbt-cloud/starburst-galaxy-config-s3.png" title="Amazon S3 connection settings in Starburst Galaxy" />
76
82
77
83
7. Click **Test connection**. This verifies that Starburst Galaxy can access your S3 bucket.
78
84
8. Click **Connect catalog** if the connection test passes.
9. On the **Set permissions** page, click **Skip**. You can add permissions later if you prefer.
87
+
9. On the **Set permissions** page, click **Skip**. You can add permissions later if you want.
83
88
10. On the **Add to cluster** page, choose the cluster you want to add the data source to from the dropdown and click **Add to cluster**.
84
-
11. Add the location privilege for your S3 bucket to your role in Starburst Galaxy. Click **Access control > Roles and privileges** on the left sidebar of the Starburst Galaxy UI. In the main body of the page, click the **accountadmin** role.
89
+
11. Add the location privilege for your S3 bucket to your role in Starburst Galaxy. Click **Access control > Roles and privileges** on the left sidebar of the Starburst Galaxy UI. Then, in the **Roles** table, click the role name **accountadmin**.
85
90
86
91
If you're using an existing Starburst Galaxy cluster and don't have access to the accountadmin role, then select a role that you do have access to.
92
+
93
+
To learn more about access control, refer to [Access control](https://docs.starburst.io/starburst-galaxy/security/access-control.html) in the Starburst Galaxy docs.
87
94
1. On the **Roles** page, click the **Privileges** tab and click **Add privilege**.
88
95
2. On the **Add privilege** page, set these options:
89
96
- **What would you like to modify privileges for?** — Choose **Location**.
90
-
- **Enter a storage location provide** — Enter the location of your S3 bucket and the folder of where the Jaffle Shop data lives. Make sure to include the `/*` at the end of the location.
97
+
- **Enter a storage location provide** — Enter the storage location of _your S3 bucket_ and the folder of where the Jaffle Shop data lives. Make sure to include the `/*` at the end of the location.
91
98
- **Create SQL** — Enable the option.
92
99
93
100
When done, click **Add privileges**.
94
101
95
102
<Lightbox src="/img/quickstarts/dbt-cloud/add-privilege.png" title="Add privilege to accountadmin role" />
96
103
97
104
## Create tables with Starburst Galaxy
98
-
To query the Jaffle Shop data with Starburst Galaxy, you need to create tables using the CSV files that you uploaded to your S3 bucket.
105
+
To query the Jaffle Shop data with Starburst Galaxy, you need to create tables using the Jaffle Shop data that you [loaded to your S3 bucket](#load-data-to-s3). You can do this (and run any SQL statement) from the [query editor](https://docs.starburst.io/starburst-galaxy/query/index.html).
99
106
100
-
1. Click **Query > Query editor** on the left sidebar of the Starburst Galaxy UI. The Query editor is now the main body of the page.
101
-
2. Configure the Query editor settings so it queries your S3 bucket. In the upper right of the editor, select your cluster in the first gray box and select your catalog in the second gray box:
107
+
1. Click **Query > Query editor** on the left sidebar of the Starburst Galaxy UI. The main body of the page is now the query editor.
108
+
2. Configure the query editor so it queries your S3 bucket. In the upper right corner of the query editor, select your cluster in the first gray box and select your catalog in the second gray box:
102
109
103
-
<Lightbox src="/img/quickstarts/dbt-cloud/starburst-galaxy-editor.png" title="Set the cluster and catalog in Query editor" />
110
+
<Lightbox src="/img/quickstarts/dbt-cloud/starburst-galaxy-editor.png" title="Set the cluster and catalog in query editor" />
104
111
105
-
3. Copy and paste these queries into the Query editor. Then **Run** each query individually.
112
+
3. Copy and paste these queries into the query editor. Then **Run** each query individually.
106
113
107
-
Replace `YOUR_S3_BUCKET_NAME` with the name of your S3 bucket you uploaded the sample data files to. These queries create a schema named `jaffle_shop` and also create the `jaffle_shop_customers`, `jaffle_shop_orders`, and `stripe_payments` tables:
114
+
Replace `YOUR_S3_BUCKET_NAME` with the name of your S3 bucket. These queries create a schema named `jaffle_shop` and also create the `jaffle_shop_customers`, `jaffle_shop_orders`, and `stripe_payments` tables:
108
115
109
116
```sql
110
117
CREATE SCHEMA jaffle_shop WITH (location='s3://YOUR_S3_BUCKET_NAME/dbt-quickstart/');
@@ -155,11 +162,11 @@ To query the Jaffle Shop data with Starburst Galaxy, you need to create tables u
155
162
156
163
);
157
164
```
158
-
4. When the queries are done, you can see the following hierarchy on the left sidebar of the Query editor:
165
+
4. When the queries are done, you can see the following hierarchy on the query editor's left sidebar:
159
166
160
-
<Lightbox src="/img/quickstarts/dbt-cloud/starburst-data-hierarchy.png" title="Hierarchy of data in Query editor" />
167
+
<Lightbox src="/img/quickstarts/dbt-cloud/starburst-data-hierarchy.png" title="Hierarchy of data in query editor" />
161
168
162
-
5. Verify that the tables were created successfully. In the Query editor, run the following queries:
169
+
5. Verify that the tables were created successfully. In the query editor, run the following queries:
163
170
164
171
```sql
165
172
select * from jaffle_shop.jaffle_shop_customers;
@@ -289,4 +296,11 @@ Later, you can connect your business intelligence (BI) tools to these views and
0 commit comments