Skip to content

Commit 9d35368

Browse files
aniketpaludevin-ai-integration[bot]
aniketpalu
and
devin-ai-integration[bot]
authored
feat: Add Oracle DB as Offline store in python sdk & operator (#6017)
* feat: Add Oracle DB as Offline store in python sdk & support in feast-operator Signed-off-by: Aniket Paluskar <[email protected]> * Added oracle db dependency from ibis-framework subgroups Signed-off-by: Aniket Paluskar <[email protected]> * Operator yaml changes Signed-off-by: Aniket Paluskar <[email protected]> * Data source writer ignored parameters, fixed Signed-off-by: Aniket Paluskar <[email protected]> * Replaced raw sql with dedicated truncate_table() to fix SQL Injection Vulnerability Signed-off-by: Aniket Paluskar <[email protected]> * Minor improvements like single db connection, removal of default creds & validation for mutually exclusive fields Signed-off-by: Aniket Paluskar <[email protected]> * Fetching pre-filtered table from db Signed-off-by: Aniket Paluskar <[email protected]> * Minor formatting changes Signed-off-by: Aniket Paluskar <[email protected]> * Added Oracle DB Offline Store documentation Signed-off-by: Aniket Paluskar <[email protected]> * Resolved import error by removing OracleSource import from the __init__ Signed-off-by: Aniket Paluskar <[email protected]> * Fixed lint error by updating secret baseline Signed-off-by: Aniket Paluskar <[email protected]> * fix: Exclude qdrant from docstring tests to avoid qdrant-client 1.17.0 EnumTypeWrapper incompatibility Signed-off-by: Aniket Paluskar <[email protected]> * Generated secret.baseline to avoid lint error Signed-off-by: Aniket Paluskar <[email protected]> * Fixed lint error Signed-off-by: Aniket Paluskar <[email protected]> * Updated .secrets.baseline Signed-off-by: Aniket Paluskar <[email protected]> * Fixed lint errors Signed-off-by: Aniket Paluskar <[email protected]> * Fixed lint errors Signed-off-by: Aniket Paluskar <[email protected]> * Update sdk/python/feast/type_map.py Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: Aniket Paluskar <[email protected]> * Updated dependency lock files Signed-off-by: Aniket Paluskar <[email protected]> * Fixed lint issues in Trino Offline Store Signed-off-by: Aniket Paluskar <[email protected]> * Updated requirements Signed-off-by: Aniket Paluskar <[email protected]> * Updated pixi.lock file Signed-off-by: Aniket Paluskar <[email protected]> * Restricted non-empty feature_views in get_historical_features() to avoid runtime errors Signed-off-by: Aniket Paluskar <[email protected]> * Removed _build_data_source_reader_for_retrieval function Signed-off-by: Aniket Paluskar <[email protected]> * Modified initial query to be _ to avoid empty string casting to Null value in Oracle DB Signed-off-by: Aniket Paluskar <[email protected]> * cast DATE to TIMESTAMP in _read_oracle_table to preserve time lost by ibis date32[day] Arrow mapping Signed-off-by: Aniket Paluskar <[email protected]> * Use single database connection for pull_latest_from_table_or_query() Signed-off-by: Aniket Paluskar <[email protected]> * Improved readibility by breaking down the code into functions Signed-off-by: Aniket Paluskar <[email protected]> * Updated .secret.baseline Signed-off-by: Aniket Paluskar <[email protected]> * Updated .secret.baseline and pixi.lock Signed-off-by: Aniket Paluskar <[email protected]> * Fixed lint issue Signed-off-by: Aniket Paluskar <[email protected]> --------- Signed-off-by: Aniket Paluskar <[email protected]> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent 94ad0e7 commit 9d35368

36 files changed

+9534
-9128
lines changed

‎.secrets.baseline‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -934,7 +934,7 @@
934934
"filename": "infra/feast-operator/api/v1/featurestore_types.go",
935935
"hashed_secret": "44e17306b837162269a410204daaa5ecee4ec22c",
936936
"is_verified": false,
937-
"line_number": 726
937+
"line_number": 727
938938
}
939939
],
940940
"infra/feast-operator/api/v1/zz_generated.deepcopy.go": [
@@ -1539,5 +1539,5 @@
15391539
}
15401540
]
15411541
},
1542-
"generated_at": "2026-03-05T15:25:10Z"
1542+
"generated_at": "2026-03-10T18:11:57Z"
15431543
}

‎docs/SUMMARY.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@
9898
* [Trino (contrib)](reference/data-sources/trino.md)
9999
* [Azure Synapse + Azure SQL (contrib)](reference/data-sources/mssql.md)
100100
* [Couchbase (contrib)](reference/data-sources/couchbase.md)
101+
* [Oracle (contrib)](reference/data-sources/oracle.md)
101102
* [Offline stores](reference/offline-stores/README.md)
102103
* [Overview](reference/offline-stores/overview.md)
103104
* [Dask](reference/offline-stores/dask.md)
@@ -112,6 +113,7 @@
112113
* [Azure Synapse + Azure SQL (contrib)](reference/offline-stores/mssql.md)
113114
* [Clickhouse (contrib)](reference/offline-stores/clickhouse.md)
114115
* [Ray (contrib)](reference/offline-stores/ray.md)
116+
* [Oracle (contrib)](reference/offline-stores/oracle.md)
115117
* [Remote Offline](reference/offline-stores/remote-offline-store.md)
116118
* [Online stores](reference/online-stores/README.md)
117119
* [Overview](reference/online-stores/overview.md)

‎docs/reference/data-sources/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,3 +57,7 @@ Please see [Data Source](../../getting-started/concepts/data-ingestion.md) for a
5757
{% content-ref url="clickhouse.md" %}
5858
[clickhouse.md](clickhouse.md)
5959
{% endcontent-ref %}
60+
61+
{% content-ref url="oracle.md" %}
62+
[oracle.md](oracle.md)
63+
{% endcontent-ref %}

‎docs/reference/data-sources/oracle.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Oracle source (contrib)
2+
3+
## Description
4+
5+
Oracle data sources are Oracle database tables.
6+
These are specified by a table reference (e.g. `"TRANSACTION_FEATURES"` or `"SCHEMA.TABLE"`).
7+
8+
## Disclaimer
9+
10+
The Oracle data source does not achieve full test coverage.
11+
Please do not assume complete stability.
12+
13+
## Examples
14+
15+
Defining an Oracle source:
16+
17+
```python
18+
from feast.infra.offline_stores.contrib.oracle_offline_store.oracle_source import (
19+
OracleSource,
20+
)
21+
22+
driver_stats_source = OracleSource(
23+
name="driver_hourly_stats",
24+
table_ref="DRIVER_HOURLY_STATS",
25+
event_timestamp_column="EVENT_TIMESTAMP",
26+
created_timestamp_column="CREATED",
27+
)
28+
```
29+
30+
**Note:** Oracle stores unquoted identifiers in uppercase. Reference columns using the casing shown by Oracle (e.g. `USER_ID` for unquoted identifiers).
31+
32+
## Supported Types
33+
34+
Oracle data sources support standard Oracle numeric, string, date, and timestamp types mapped through the ibis Oracle backend.
35+
For a comparison against other batch data sources, please see [here](overview.md#functionality-matrix).
36+

‎docs/reference/offline-stores/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,3 +49,7 @@ Please see [Offline Store](../../getting-started/components/offline-store.md) fo
4949
{% content-ref url="ray.md" %}
5050
[ray.md](ray.md)
5151
{% endcontent-ref %}
52+
53+
{% content-ref url="oracle.md" %}
54+
[oracle.md](oracle.md)
55+
{% endcontent-ref %}

‎docs/reference/offline-stores/oracle.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Oracle offline store (contrib)
2+
3+
## Description
4+
5+
The Oracle offline store provides support for reading [OracleSources](../data-sources/oracle.md).
6+
* Entity dataframes can be provided as a SQL query or as a Pandas dataframe.
7+
* Uses the [ibis](https://ibis-project.org/) Oracle backend (`ibis.oracle`) for all database interactions.
8+
* Only one of `service_name`, `sid`, or `dsn` may be set in the configuration.
9+
10+
## Disclaimer
11+
12+
The Oracle offline store does not achieve full test coverage.
13+
Please do not assume complete stability.
14+
15+
## Getting started
16+
17+
Install the Oracle extras:
18+
19+
```bash
20+
pip install 'feast[oracle]'
21+
```
22+
23+
## Example
24+
25+
{% code title="feature_store.yaml" %}
26+
```yaml
27+
project: my_project
28+
registry: data/registry.db
29+
provider: local
30+
offline_store:
31+
type: oracle
32+
host: DB_HOST
33+
port: 1521
34+
user: DB_USERNAME
35+
password: DB_PASSWORD
36+
service_name: ORCL
37+
online_store:
38+
path: data/online_store.db
39+
```
40+
{% endcode %}
41+
42+
Connection can alternatively use `sid` or `dsn` instead of `service_name`:
43+
44+
```yaml
45+
# Using SID
46+
offline_store:
47+
type: oracle
48+
host: DB_HOST
49+
port: 1521
50+
user: DB_USERNAME
51+
password: DB_PASSWORD
52+
sid: ORCL
53+
54+
# Using DSN
55+
offline_store:
56+
type: oracle
57+
host: DB_HOST
58+
port: 1521
59+
user: DB_USERNAME
60+
password: DB_PASSWORD
61+
dsn: "(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=DB_HOST)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCL)))"
62+
```
63+
64+
### Configuration reference
65+
66+
| Parameter | Required | Default | Description |
67+
| :------------- | :------- | :---------- | :------------------------------------------------------- |
68+
| `type` | yes | — | Must be set to `oracle` |
69+
| `user` | yes | — | Oracle database user |
70+
| `password` | yes | — | Oracle database password |
71+
| `host` | no | `localhost` | Oracle database host |
72+
| `port` | no | `1521` | Oracle database port |
73+
| `service_name` | no | — | Oracle service name (mutually exclusive with sid and dsn) |
74+
| `sid` | no | — | Oracle SID (mutually exclusive with service_name and dsn) |
75+
| `database` | no | — | Oracle database name |
76+
| `dsn` | no | — | Oracle DSN string (mutually exclusive with service_name and sid) |
77+
78+
## Functionality Matrix
79+
80+
The set of functionality supported by offline stores is described in detail [here](overview.md#functionality).
81+
Below is a matrix indicating which functionality is supported by the Oracle offline store.
82+
83+
| | Oracle |
84+
| :----------------------------------------------------------------- | :----- |
85+
| `get_historical_features` (point-in-time correct join) | yes |
86+
| `pull_latest_from_table_or_query` (retrieve latest feature values) | yes |
87+
| `pull_all_from_table_or_query` (retrieve a saved dataset) | yes |
88+
| `offline_write_batch` (persist dataframes to offline store) | yes |
89+
| `write_logged_features` (persist logged features to offline store) | yes |
90+
91+
Below is a matrix indicating which functionality is supported by `OracleRetrievalJob`.
92+
93+
| | Oracle |
94+
| ----------------------------------------------------- | ------ |
95+
| export to dataframe | yes |
96+
| export to arrow table | yes |
97+
| export to arrow batches | no |
98+
| export to SQL | no |
99+
| export to data lake (S3, GCS, etc.) | no |
100+
| export to data warehouse | no |
101+
| export as Spark dataframe | no |
102+
| local execution of Python-based on-demand transforms | yes |
103+
| remote execution of Python-based on-demand transforms | no |
104+
| persist results in the offline store | yes |
105+
| preview the query plan before execution | no |
106+
| read partitioned data | no |
107+
108+
To compare this set of functionality against other offline stores, please see the full [functionality matrix](overview.md#functionality-matrix).
109+

‎infra/feast-operator/api/v1/featurestore_types.go‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -402,7 +402,7 @@ var ValidOfflineStoreFilePersistenceTypes = []string{
402402
// OfflineStoreDBStorePersistence configures the DB store persistence for the offline store service
403403
type OfflineStoreDBStorePersistence struct {
404404
// Type of the persistence type you want to use.
405-
// +kubebuilder:validation:Enum=snowflake.offline;bigquery;redshift;spark;postgres;trino;athena;mssql;couchbase.offline;clickhouse;ray
405+
// +kubebuilder:validation:Enum=snowflake.offline;bigquery;redshift;spark;postgres;trino;athena;mssql;couchbase.offline;clickhouse;ray;oracle
406406
Type string `json:"type"`
407407
// Data store parameters should be placed as-is from the "feature_store.yaml" under the secret key. "registry_type" & "type" fields should be removed.
408408
SecretRef corev1.LocalObjectReference `json:"secretRef"`
@@ -422,6 +422,7 @@ var ValidOfflineStoreDBStorePersistenceTypes = []string{
422422
"couchbase.offline",
423423
"clickhouse",
424424
"ray",
425+
"oracle",
425426
}
426427

427428
// OnlineStore configures the online store service

‎infra/feast-operator/bundle/manifests/feast.dev_featurestores.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1631,6 +1631,7 @@ spec:
16311631
- couchbase.offline
16321632
- clickhouse
16331633
- ray
1634+
- oracle
16341635
type: string
16351636
required:
16361637
- secretRef
@@ -7341,6 +7342,7 @@ spec:
73417342
- couchbase.offline
73427343
- clickhouse
73437344
- ray
7345+
- oracle
73447346
type: string
73457347
required:
73467348
- secretRef

‎infra/feast-operator/config/crd/bases/feast.dev_featurestores.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1631,6 +1631,7 @@ spec:
16311631
- couchbase.offline
16321632
- clickhouse
16331633
- ray
1634+
- oracle
16341635
type: string
16351636
required:
16361637
- secretRef
@@ -7341,6 +7342,7 @@ spec:
73417342
- couchbase.offline
73427343
- clickhouse
73437344
- ray
7345+
- oracle
73447346
type: string
73457347
required:
73467348
- secretRef

‎infra/feast-operator/dist/install.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1639,6 +1639,7 @@ spec:
16391639
- couchbase.offline
16401640
- clickhouse
16411641
- ray
1642+
- oracle
16421643
type: string
16431644
required:
16441645
- secretRef
@@ -7349,6 +7350,7 @@ spec:
73497350
- couchbase.offline
73507351
- clickhouse
73517352
- ray
7353+
- oracle
73527354
type: string
73537355
required:
73547356
- secretRef

0 commit comments

Comments
 (0)