Skip to content

Commit c965a77

Browse files
woopShu Heng
woop
authored and
Shu Heng
committed
GitBook: [master] 5 pages modified
1 parent 50a9992 commit c965a77

File tree

5 files changed

+220
-50
lines changed

5 files changed

+220
-50
lines changed

‎docs/SUMMARY.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
* [Overview](installing-feast/overview.md)
1212
* [Docker Compose](installing-feast/docker-compose.md)
1313
* [Google Kubernetes Engine \(GKE\)](installing-feast/gke.md)
14+
* [Troubleshooting](installing-feast/troubleshooting.md)
1415

1516
## Using Feast
1617

‎docs/installing-feast/docker-compose.md‎

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ cp .env.sample .env
4646

4747
## 2. Docker Compose for Online Serving Only
4848

49+
### 2.1 Start Feast \(without batch retrieval support\)
50+
4951
If you do not require batch serving, then its possible to simply bring up Feast:
5052

5153
```javascript
@@ -56,13 +58,15 @@ A Jupyter Notebook environment is now available to use Feast:
5658

5759
[http://localhost:8888/tree/feast/examples](http://localhost:8888/tree/feast/examples)
5860

59-
## 2. Docker Compose for Online and Batch Serving
61+
## 3. Docker Compose for Online and Batch Serving
6062

6163
{% hint style="info" %}
6264
Batch serving requires Google Cloud Storage to function, specifically Google Cloud Storage \(GCP\) and BigQuery.
6365
{% endhint %}
6466

65-
Create a [service account ](https://cloud.google.com/iam/docs/creating-managing-service-accounts)from the GCP console and copy it to the `gcp-service-accounts` folder:
67+
### 3.1 Set up Google Cloud Platform
68+
69+
Create a [service account ](https://cloud.google.com/iam/docs/creating-managing-service-accounts)from the GCP console and copy it to the `infra/docker-compose/gcp-service-accounts` folder:
6670

6771
```javascript
6872
cp my-service-account.json ${FEAST_HOME_DIR}/infra/docker-compose/gcp-service-accounts
@@ -74,28 +78,32 @@ Create a Google Cloud Storage bucket. Make sure that your service account above
7478
gsutil mb gs://my-feast-staging-bucket
7579
```
7680

77-
### 2.1 Configure .env
81+
### 3.2 Configure .env
7882

7983
Configure the `.env` file based on your environment. At the very least you have to modify:
8084

81-
* **FEAST\_CORE\_GCP\_SERVICE\_ACCOUNT\_KEY:** This should be your service account file name, for example `key.json`.
82-
* **FEAST\_BATCH\_SERVING\_GCP\_SERVICE\_ACCOUNT\_KEY:** This should be your service account file name, for example `key.json`.
83-
* **FEAST\_JUPYTER\_GCP\_SERVICE\_ACCOUNT\_KEY:** This should be your service account file name, for example `key.json`.
84-
* **FEAST\_JOB\_STAGING\_LOCATION:** Google Cloud Storage bucket that Feast will use to stage data exports and batch retrieval requests, for example `gs://your-gcs-bucket/staging`
85+
| Parameter | Description |
86+
| :--- | :--- |
87+
| FEAST\_CORE\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json`. |
88+
| FEAST\_BATCH\_SERVING\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json` |
89+
| FEAST\_JUPYTER\_GCP\_SERVICE\_ACCOUNT\_KEY | This should be your service account file name, for example `key.json` |
90+
| FEAST\_JOB\_STAGING\_LOCATION | Google Cloud Storage bucket that Feast will use to stage data exports and batch retrieval requests, for example `gs://your-gcs-bucket/staging` |
8591

86-
### 2.2 Configure .bq-store.yml
92+
### 3.3 Configure .bq-store.yml
8793

8894
We will also need to configure the `bq-store.yml` file inside `infra/docker-compose/serving/` to configure the BigQuery storage configuration as well as the feature sets that the store subscribes to. At a minimum you will need to set:
8995

90-
* **project\_id:** This is you [GCP project Id](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
91-
* **dataset\_id:** This is the name of the BigQuery dataset that tables will be created in. Each feature set will have one table in BigQuery.
96+
| Parameter | Description |
97+
| :--- | :--- |
98+
| bigquery\_config.project\_id | This is you [GCP project Id](https://cloud.google.com/resource-manager/docs/creating-managing-projects). |
99+
| bigquery\_config.dataset\_id | This is the name of the BigQuery dataset that tables will be created in. Each feature set will have one table in BigQuery. |
92100

93-
### 2.3 Start Feast with batch retrieval support
101+
### 3.4 Start Feast \(with batch retrieval support\)
94102

95103
Start Feast:
96104

97105
```javascript
98-
docker-compose -f docker-compose.yml -f docker-compose.batch.yml up -d
106+
docker-compose up -d
99107
```
100108

101109
A Jupyter Notebook environment is now available to use Feast:

‎docs/installing-feast/gke.md‎

Lines changed: 30 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,6 @@ This guide will install Feast into a Kubernetes cluster on GCP. It assumes that
1313
This guide requires [Google Cloud Platform](https://cloud.google.com/) for installation.
1414

1515
* [BigQuery](https://cloud.google.com/bigquery/) is used for storing historical features.
16-
* [Cloud Dataflow](https://cloud.google.com/dataflow/) is used for running data ingestion jobs.
1716
* [Google Cloud Storage](https://cloud.google.com/storage/) is used for intermediate data storage.
1817
{% endhint %}
1918

@@ -34,38 +33,26 @@ export FEAST_GCP_ZONE=us-central1-a
3433
export FEAST_BIGQUERY_DATASET_ID=feast
3534
export FEAST_GCS_BUCKET=${FEAST_GCP_PROJECT_ID}_feast_bucket
3635
export FEAST_GKE_CLUSTER_NAME=feast
37-
export FEAST_S_ACCOUNT_NAME=feast-sa
36+
export FEAST_SERVICE_ACCOUNT_NAME=feast-sa
3837
```
3938

40-
Create a Google Cloud Storage bucket for Feast to stage data during exports:
39+
Create a Google Cloud Storage bucket for Feast to stage batch data exports:
4140

4241
```bash
4342
gsutil mb gs://${FEAST_GCS_BUCKET}
4443
```
4544

46-
Create a BigQuery dataset for storing historical features:
47-
48-
```bash
49-
bq mk ${FEAST_BIGQUERY_DATASET_ID}
50-
```
51-
5245
Create the service account that Feast will run as:
5346

5447
```bash
55-
gcloud iam service-accounts create ${FEAST_S_ACCOUNT_NAME}
48+
gcloud iam service-accounts create ${FEAST_SERVICE_ACCOUNT_NAME}
5649

5750
gcloud projects add-iam-policy-binding ${FEAST_GCP_PROJECT_ID} \
58-
--member serviceAccount:${FEAST_S_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com \
51+
--member serviceAccount:${FEAST_SERVICE_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com \
5952
--role roles/editor
6053

6154
gcloud iam service-accounts keys create key.json --iam-account \
62-
${FEAST_S_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com
63-
```
64-
65-
Ensure that [Dataflow API is enabled](https://console.cloud.google.com/apis/api/dataflow.googleapis.com/overview):
66-
67-
```bash
68-
gcloud services enable dataflow.googleapis.com
55+
${FEAST_SERVICE_ACCOUNT_NAME}@${FEAST_GCP_PROJECT_ID}.iam.gserviceaccount.com
6956
```
7057

7158
## 2. Set up a Kubernetes \(GKE\) cluster
@@ -87,7 +74,7 @@ Create a secret in the GKE cluster based on your local key `key.json`:
8774
kubectl create secret generic feast-gcp-service-account --from-file=key.json
8875
```
8976

90-
For this guide we will use `NodePort` for exposing Feast services. In order to do so, we must find an internal IP of at least one GKE node.
77+
For this guide we will use `NodePort` for exposing Feast services. In order to do so, we must find an External IP of at least one GKE node. This should be a public IP.
9178

9279
```bash
9380
export FEAST_IP=$(kubectl describe nodes | grep ExternalIP | awk '{print $2}' | head -n 1)
@@ -96,18 +83,6 @@ export FEAST_ONLINE_SERVING_URL=${FEAST_IP}:32091
9683
export FEAST_BATCH_SERVING_URL=${FEAST_IP}:32092
9784
```
9885

99-
Confirm that you are able to access this node \(please make sure that no firewall rules are preventing access to these ports\):
100-
101-
```bash
102-
ping $FEAST_IP
103-
```
104-
105-
```bash
106-
PING 10.123.114.11 (10.203.164.22) 56(84) bytes of data.
107-
64 bytes from 10.123.114.11: icmp_seq=1 ttl=63 time=54.2 ms
108-
64 bytes from 10.123.114.11: icmp_seq=2 ttl=63 time=51.2 ms
109-
```
110-
11186
Add firewall rules to open up ports on your Google Cloud Platform project:
11287

11388
```bash
@@ -170,25 +145,29 @@ cp values.yaml my-feast-values.yaml
170145
Update `my-feast-values.yaml` based on your GCP and GKE environment.
171146

172147
* Required fields are paired with comments which indicate whether they need to be replaced.
173-
* All occurrences of `EXTERNAL_IP` should be replaced with either your domain name or the IP stored in `$FEAST_IP`.
148+
* All occurrences of `EXTERNAL_IP` should be replaced with either a domain pointing to a load balancer for the cluster or the IP stored in `$FEAST_IP`.
174149
* Replace all occurrences of `YOUR_BUCKET_NAME` with your bucket name stored in `$FEAST_GCS_BUCKET`
150+
* Change `feast-serving-batch.store.yaml.bigquery_config.project_id` to your GCP project Id.
151+
* Change `feast-serving-batch.store.yaml.bigquery_config.dataset_id` to the BigQuery dataset that Feast should use.
175152

176153
Install the Feast Helm chart:
177154

178155
```bash
179156
helm install --name feast -f my-feast-values.yaml .
180157
```
181158

182-
Ensure that the system comes online. This will take a few minutes
159+
Ensure that the system comes online. This will take a few minutes.
183160

184161
```bash
185-
watch kubectl get pods
162+
kubectl get pods
186163
```
187164

165+
There may be pod restarts while waiting for Kafka to come online.
166+
188167
```bash
189168
NAME READY STATUS RESTARTS AGE
190-
pod/feast-feast-core-666fd46db4-l58l6 1/1 Running 0 5m
191-
pod/feast-feast-serving-online-84d99ddcbd 1/1 Running 0 6m
169+
pod/feast-feast-core-666fd46db4-l58l6 1/1 Running 2 5m
170+
pod/feast-feast-serving-online-84d99ddcbd 1/1 Running 3 6m
192171
pod/feast-kafka-0 1/1 Running 0 3m
193172
pod/feast-kafka-1 1/1 Running 0 4m
194173
pod/feast-kafka-2 1/1 Running 0 4m
@@ -204,7 +183,7 @@ pod/feast-zookeeper-2 1/1 Running 0 5m
204183
Install the Python SDK using pip:
205184

206185
```bash
207-
pip install -e ${FEAST_HOME_DIR}/sdk/python
186+
pip install feast
208187
```
209188

210189
Configure the Feast Python SDK:
@@ -214,5 +193,19 @@ feast config set core_url ${FEAST_CORE_URL}
214193
feast config set serving_url ${FEAST_ONLINE_SERVING_URL}
215194
```
216195

196+
Test whether you are able to connect to Feast Core
197+
198+
```text
199+
feast projects list
200+
```
201+
202+
Should print an empty list:
203+
204+
```text
205+
NAME
206+
```
207+
217208
That's it! You can now start to use Feast!
218209

210+
Please see our [examples](https://github.com/gojek/feast/blob/master/examples/) to get started.
211+

‎docs/installing-feast/overview.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This installation guide will demonstrate three ways of installing Feast:
88
* Does not officially support a production job manager like Dataflow
99
* \*\*\*\*[**Google Kubernetes Engine**](gke.md)**:**
1010
* Recommended way to install Feast for production use.
11-
* The guide has dependencies on BigQuery, Dataflow, and Google Cloud Storage.
11+
* The guide has dependencies on BigQuery, and Google Cloud Storage.
1212

1313

1414

‎docs/installing-feast/troubleshooting.md‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Troubleshooting
2+
3+
If at any point in time you cannot resolve a problem, please see the [Getting Help](../getting-help.md) section for reaching out to the Feast community.
4+
5+
## How can I verify that all services are operational?
6+
7+
### Docker Compose
8+
9+
The containers should be in an `up` state:
10+
11+
```text
12+
docker ps
13+
```
14+
15+
```text
16+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
17+
d7447205bced jupyter/datascience-notebook:latest "tini -g -- start-no…" 2 minutes ago Up 2 minutes 0.0.0.0:8888->8888/tcp feast_jupyter_1
18+
8e49dbe81b92 gcr.io/kf-feast/feast-serving:latest "java -Xms1024m -Xmx…" 2 minutes ago Up 5 seconds 0.0.0.0:6567->6567/tcp feast_batch-serving_1
19+
b859494bd33a gcr.io/kf-feast/feast-serving:latest "java -jar /opt/feas…" 2 minutes ago Up About a minute 0.0.0.0:6566->6566/tcp feast_online-serving_1
20+
5c4962811767 gcr.io/kf-feast/feast-core:latest "java -jar /opt/feas…" 2 minutes ago Up 2 minutes 0.0.0.0:6565->6565/tcp feast_core_1
21+
1ba7239e0ae0 confluentinc/cp-kafka:5.2.1 "/etc/confluent/dock…" 2 minutes ago Up 2 minutes 0.0.0.0:9092->9092/tcp, 0.0.0.0:9094->9094/tcp feast_kafka_1
22+
e2779672735c confluentinc/cp-zookeeper:5.2.1 "/etc/confluent/dock…" 2 minutes ago Up 2 minutes 2181/tcp, 2888/tcp, 3888/tcp feast_zookeeper_1
23+
39ac26f5c709 postgres:12-alpine "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 5432/tcp feast_db_1
24+
3c4ee8616096 redis:5-alpine "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:6379->6379/tcp feast_redis_1
25+
```
26+
27+
### Google Kubernetes Engine
28+
29+
All services should either be in a `running` state or `complete`state:
30+
31+
```text
32+
kubectl get pods
33+
```
34+
35+
```text
36+
NAME READY STATUS RESTARTS AGE
37+
feast-feast-core-5ff566f946-4wlbh 1/1 Running 1 32m
38+
feast-feast-serving-batch-848d74587b-96hq6 1/1 Running 2 32m
39+
feast-feast-serving-online-df69755d5-fml8v 1/1 Running 2 32m
40+
feast-kafka-0 1/1 Running 1 32m
41+
feast-kafka-1 1/1 Running 0 30m
42+
feast-kafka-2 1/1 Running 0 29m
43+
feast-kafka-config-3e860262-zkzr8 0/1 Completed 0 32m
44+
feast-postgresql-0 1/1 Running 0 32m
45+
feast-prometheus-statsd-exporter-554db85b8d-r4hb8 1/1 Running 0 32m
46+
feast-redis-master-0 1/1 Running 0 32m
47+
feast-zookeeper-0 1/1 Running 0 32m
48+
feast-zookeeper-1 1/1 Running 0 32m
49+
feast-zookeeper-2 1/1 Running 0 31m
50+
```
51+
52+
## How can I verify that I can connect to all services?
53+
54+
First find the `IP:Port` combination of your services.
55+
56+
### **Docker Compose \(from inside the docker cluster\)**
57+
58+
You will probably need to connect using the hostnames of services and standard Feast ports:
59+
60+
```bash
61+
export FEAST_CORE_URL=core:6565
62+
export FEAST_ONLINE_SERVING_URL=online-serving:6566
63+
export FEAST_BATCH_SERVING_URL=batch-serving:6567
64+
```
65+
66+
### **Docker Compose \(from outside the docker cluster\)**
67+
68+
You will probably need to connect using `localhost` and standard ports:
69+
70+
```bash
71+
export FEAST_CORE_URL=localhost:6565
72+
export FEAST_ONLINE_SERVING_URL=localhost:6566
73+
export FEAST_BATCH_SERVING_URL=localhost:6567
74+
```
75+
76+
### **Google Kubernetes Engine \(GKE\)**
77+
78+
You will need to find the external IP of one of the nodes as well as the NodePorts. Please make sure that your firewall is open for these ports:
79+
80+
```bash
81+
export FEAST_IP=$(kubectl describe nodes | grep ExternalIP | awk '{print $2}' | head -n 1)
82+
export FEAST_CORE_URL=${FEAST_IP}:32090
83+
export FEAST_ONLINE_SERVING_URL=${FEAST_IP}:32091
84+
export FEAST_BATCH_SERVING_URL=${FEAST_IP}:32092
85+
```
86+
87+
`netcat`, `telnet`, or even `curl` can be used to test whether all services are available and ports are open, but `grpc_cli` is the most powerful. It can be installed from [here](https://github.com/grpc/grpc/blob/master/doc/command_line_tool.md).
88+
89+
### Testing Feast Core:
90+
91+
```bash
92+
grpc_cli ls ${FEAST_CORE_URL} feast.core.CoreService
93+
```
94+
95+
```text
96+
GetFeastCoreVersion
97+
GetFeatureSet
98+
ListFeatureSets
99+
ListStores
100+
ApplyFeatureSet
101+
UpdateStore
102+
CreateProject
103+
ArchiveProject
104+
ListProjects
105+
```
106+
107+
### Testing Feast Batch Serving and Online Serving
108+
109+
```bash
110+
grpc_cli ls ${FEAST_BATCH_SERVING_URL} feast.serving.ServingService
111+
```
112+
113+
```text
114+
GetFeastServingInfo
115+
GetOnlineFeatures
116+
GetBatchFeatures
117+
GetJob
118+
```
119+
120+
```bash
121+
grpc_cli ls ${FEAST_ONLINE_SERVING_URL} feast.serving.ServingService
122+
```
123+
124+
```text
125+
GetFeastServingInfo
126+
GetOnlineFeatures
127+
GetBatchFeatures
128+
GetJob
129+
```
130+
131+
## How can I print logs from the Feast Services?
132+
133+
Feast will typically have three services that you need to monitor if something goes wrong.
134+
135+
* Feast Core
136+
* Feast Serving \(Online\)
137+
* Feast Serving \(Batch\)
138+
139+
In order to print the logs from these services, please run the commands below.
140+
141+
### Docker Compose
142+
143+
```text
144+
docker logs -f feast_core_1
145+
```
146+
147+
```text
148+
docker logs -f feast_batch-serving_1
149+
```
150+
151+
```text
152+
docker logs -f feast_online-serving_1
153+
```
154+
155+
### Google Kubernetes Engine
156+
157+
```text
158+
kubectl logs $(kubectl get pods | grep feast-core | awk '{print $1}')
159+
```
160+
161+
```text
162+
kubectl logs $(kubectl get pods | grep feast-serving-batch | awk '{print $1}')
163+
```
164+
165+
```text
166+
kubectl logs $(kubectl get pods | grep feast-serving-online | awk '{print $1}')
167+
```
168+

0 commit comments

Comments
 (0)