Skip to content

Commit 8f4ba7c

Browse files
yxtayyxtay
authored
fix: use docker compose (#1905)
Signed-off-by: YuXuan Tay <[email protected]>
1 parent 75e0033 commit 8f4ba7c

File tree

9 files changed

+163
-133
lines changed

9 files changed

+163
-133
lines changed

‎docs/user_guide/storing/doc_store/store_s3.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ When you want to use your [`DocList`][docarray.DocList] in another place, you ca
1212
## Push & pull
1313
To use the store [`DocList`][docarray.DocList] on S3, you need to pass an S3 path to the function starting with `'s3://'`.
1414

15-
In the following demo, we use `MinIO` as a local S3 service. You could use the following docker-compose file to start the service in a Docker container.
15+
In the following demo, we use `MinIO` as a local S3 service. You could use the following docker compose file to start the service in a Docker container.
1616

1717
```yaml
1818
version: "3"
@@ -26,7 +26,7 @@ services:
2626
```
2727
Save the above file as `docker-compose.yml` and run the following line in the same folder as the file.
2828
```cmd
29-
docker-compose up
29+
docker compose up
3030
```
3131

3232
```python

‎docs/user_guide/storing/index_elastic.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -45,13 +45,17 @@ from docarray.index import ElasticDocIndex # or ElasticV7DocIndex
4545
from docarray.typing import NdArray
4646
import numpy as np
4747

48+
4849
# Define the document schema.
4950
class MyDoc(BaseDoc):
50-
title: str
51+
title: str
5152
embedding: NdArray[128]
5253

54+
5355
# Create dummy documents.
54-
docs = DocList[MyDoc](MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10))
56+
docs = DocList[MyDoc](
57+
MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10)
58+
)
5559

5660
# Initialize a new ElasticDocIndex instance and add the documents to the index.
5761
doc_index = ElasticDocIndex[MyDoc](index_name='my_index')
@@ -67,7 +71,7 @@ retrieved_docs = doc_index.find(query, search_field='embedding', limit=10)
6771
## Initialize
6872

6973

70-
You can use docker-compose to create a local Elasticsearch service with the following `docker-compose.yml`.
74+
You can use docker compose to create a local Elasticsearch service with the following `docker-compose.yml`.
7175

7276
```yaml
7377
version: "3.3"
@@ -91,7 +95,7 @@ networks:
9195
Run the following command in the folder of the above `docker-compose.yml` to start the service:
9296

9397
```bash
94-
docker-compose up
98+
docker compose up
9599
```
96100

97101
### Schema definition
@@ -225,9 +229,7 @@ You can also search for multiple documents at once, in a batch, using the [`find
225229

226230
```python
227231
# create some query Documents
228-
queries = DocList[SimpleDoc](
229-
SimpleDoc(tensor=np.random.rand(128)) for i in range(3)
230-
)
232+
queries = DocList[SimpleDoc](SimpleDoc(tensor=np.random.rand(128)) for i in range(3))
231233
232234
# find similar documents
233235
matches, scores = doc_index.find_batched(queries, search_field='tensor', limit=5)

‎docs/user_guide/storing/index_milvus.md‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,17 @@ from docarray.typing import NdArray
2727
from pydantic import Field
2828
import numpy as np
2929

30+
3031
# Define the document schema.
3132
class MyDoc(BaseDoc):
32-
title: str
33+
title: str
3334
embedding: NdArray[128] = Field(is_embedding=True)
3435

36+
3537
# Create dummy documents.
36-
docs = DocList[MyDoc](MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10))
38+
docs = DocList[MyDoc](
39+
MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10)
40+
)
3741

3842
# Initialize a new MilvusDocumentIndex instance and add the documents to the index.
3943
doc_index = MilvusDocumentIndex[MyDoc](index_name='tmp_index_1')
@@ -55,7 +59,7 @@ wget https://github.com/milvus-io/milvus/releases/download/v2.2.11/milvus-standa
5559

5660
And start Milvus by running:
5761
```shell
58-
sudo docker-compose up -d
62+
sudo docker compose up -d
5963
```
6064

6165
Learn more on [Milvus documentation](https://milvus.io/docs/install_standalone-docker.md).
@@ -142,10 +146,12 @@ Now that you have a Document Index, you can add data to it, using the [`index()`
142146
import numpy as np
143147
from docarray import DocList
144148

149+
145150
class MyDoc(BaseDoc):
146-
title: str
151+
title: str
147152
embedding: NdArray[128] = Field(is_embedding=True)
148153

154+
149155
doc_index = MilvusDocumentIndex[MyDoc](index_name='tmp_index_5')
150156

151157
# create some random data
@@ -273,7 +279,9 @@ class Book(BaseDoc):
273279
embedding: NdArray[10] = Field(is_embedding=True)
274280

275281

276-
books = DocList[Book]([Book(price=i * 10, embedding=np.random.rand(10)) for i in range(10)])
282+
books = DocList[Book](
283+
[Book(price=i * 10, embedding=np.random.rand(10)) for i in range(10)]
284+
)
277285
book_index = MilvusDocumentIndex[Book](index_name='tmp_index_6')
278286
book_index.index(books)
279287

@@ -312,8 +320,11 @@ class SimpleSchema(BaseDoc):
312320
price: int
313321
embedding: NdArray[128] = Field(is_embedding=True)
314322

323+
315324
# Create dummy documents.
316-
docs = DocList[SimpleSchema](SimpleSchema(price=i, embedding=np.random.rand(128)) for i in range(10))
325+
docs = DocList[SimpleSchema](
326+
SimpleSchema(price=i, embedding=np.random.rand(128)) for i in range(10)
327+
)
317328

318329
doc_index = MilvusDocumentIndex[SimpleSchema](index_name='tmp_index_7')
319330
doc_index.index(docs)
@@ -407,7 +418,9 @@ You can pass any of the above as keyword arguments to the `__init__()` method or
407418

408419
```python
409420
class SimpleDoc(BaseDoc):
410-
tensor: NdArray[128] = Field(is_embedding=True, index_type='IVF_FLAT', metric_type='L2')
421+
tensor: NdArray[128] = Field(
422+
is_embedding=True, index_type='IVF_FLAT', metric_type='L2'
423+
)
411424

412425

413426
doc_index = MilvusDocumentIndex[SimpleDoc](index_name='tmp_index_10')

‎docs/user_guide/storing/index_qdrant.md‎

Lines changed: 31 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,17 @@ from docarray.index import QdrantDocumentIndex
2222
from docarray.typing import NdArray
2323
import numpy as np
2424

25+
2526
# Define the document schema.
2627
class MyDoc(BaseDoc):
27-
title: str
28+
title: str
2829
embedding: NdArray[128]
2930

31+
3032
# Create dummy documents.
31-
docs = DocList[MyDoc](MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10))
33+
docs = DocList[MyDoc](
34+
MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10)
35+
)
3236

3337
# Initialize a new QdrantDocumentIndex instance and add the documents to the index.
3438
doc_index = QdrantDocumentIndex[MyDoc](host='localhost')
@@ -46,7 +50,7 @@ You can initialize [QdrantDocumentIndex][docarray.index.backends.qdrant.QdrantDo
4650

4751
**Connecting to a local Qdrant instance running as a Docker container**
4852

49-
You can use docker-compose to create a local Qdrant service with the following `docker-compose.yml`.
53+
You can use docker compose to create a local Qdrant service with the following `docker-compose.yml`.
5054

5155
```yaml
5256
version: '3.8'
@@ -66,7 +70,7 @@ services:
6670
Run the following command in the folder of the above `docker-compose.yml` to start the service:
6771

6872
```bash
69-
docker-compose up
73+
docker compose up
7074
```
7175

7276
Next, you can create a [QdrantDocumentIndex][docarray.index.backends.qdrant.QdrantDocumentIndex] instance using:
@@ -89,7 +93,7 @@ doc_index = QdrantDocumentIndex[MyDoc](qdrant_config)
8993
**Connecting to Qdrant Cloud service**
9094
```python
9195
qdrant_config = QdrantDocumentIndex.DBConfig(
92-
"https://YOUR-CLUSTER-URL.aws.cloud.qdrant.io",
96+
"https://YOUR-CLUSTER-URL.aws.cloud.qdrant.io",
9397
api_key="<your-api-key>",
9498
)
9599
doc_index = QdrantDocumentIndex[MyDoc](qdrant_config)
@@ -317,9 +321,7 @@ book_index = QdrantDocumentIndex[Book]()
317321
book_index.index(books)
318322
319323
# filter for books that are cheaper than 29 dollars
320-
query = rest.Filter(
321-
must=[rest.FieldCondition(key='price', range=rest.Range(lt=29))]
322-
)
324+
query = rest.Filter(must=[rest.FieldCondition(key='price', range=rest.Range(lt=29))])
323325
cheap_books = book_index.filter(filter_query=query)
324326
325327
assert len(cheap_books) == 3
@@ -372,24 +374,26 @@ class SimpleDoc(BaseDoc):
372374
373375
doc_index = QdrantDocumentIndex[SimpleDoc](host='localhost')
374376
index_docs = [
375-
SimpleDoc(id=f'{i}', tens=np.ones(10) * i, num=int(i / 2), text=f'Lorem ipsum {int(i/2)}')
377+
SimpleDoc(
378+
id=f'{i}', tens=np.ones(10) * i, num=int(i / 2), text=f'Lorem ipsum {int(i/2)}'
379+
)
376380
for i in range(10)
377381
]
378382
doc_index.index(index_docs)
379383
380384
find_query = np.ones(10)
381385
text_search_query = 'ipsum 1'
382386
filter_query = rest.Filter(
383-
must=[
384-
rest.FieldCondition(
385-
key='num',
386-
range=rest.Range(
387-
gte=1,
388-
lt=5,
389-
),
390-
)
391-
]
392-
)
387+
must=[
388+
rest.FieldCondition(
389+
key='num',
390+
range=rest.Range(
391+
gte=1,
392+
lt=5,
393+
),
394+
)
395+
]
396+
)
393397
394398
query = (
395399
doc_index.build_query()
@@ -437,6 +441,8 @@ import numpy as np
437441
from docarray import BaseDoc, DocList
438442
from docarray.typing import NdArray
439443
from docarray.index import QdrantDocumentIndex
444+
445+
440446
class MyDoc(BaseDoc):
441447
text: str
442448
embedding: NdArray[128]
@@ -445,7 +451,12 @@ class MyDoc(BaseDoc):
445451
Now, we can instantiate our Index and add some data:
446452
```python
447453
docs = DocList[MyDoc](
448-
[MyDoc(embedding=np.random.rand(10), text=f'I am the first version of Document {i}') for i in range(100)]
454+
[
455+
MyDoc(
456+
embedding=np.random.rand(10), text=f'I am the first version of Document {i}'
457+
)
458+
for i in range(100)
459+
]
449460
)
450461
index = QdrantDocumentIndex[MyDoc]()
451462
index.index(docs)

‎docs/user_guide/storing/index_weaviate.md‎

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,17 @@ from docarray.typing import NdArray
2727
from pydantic import Field
2828
import numpy as np
2929

30+
3031
# Define the document schema.
3132
class MyDoc(BaseDoc):
32-
title: str
33+
title: str
3334
embedding: NdArray[128] = Field(is_embedding=True)
3435

36+
3537
# Create dummy documents.
36-
docs = DocList[MyDoc](MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10))
38+
docs = DocList[MyDoc](
39+
MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10)
40+
)
3741

3842
# Initialize a new WeaviateDocumentIndex instance and add the documents to the index.
3943
doc_index = WeaviateDocumentIndex[MyDoc]()
@@ -59,7 +63,7 @@ There are multiple ways to start a Weaviate instance, depending on your use case
5963
| ----- | ----- | ----- | ----- |
6064
| **Weaviate Cloud Services (WCS)** | Development and production | Limited | **Recommended for most users** |
6165
| **Embedded Weaviate** | Experimentation | Limited | Experimental (as of Apr 2023) |
62-
| **Docker-Compose** | Development | Yes | **Recommended for development + customizability** |
66+
| **Docker Compose** | Development | Yes | **Recommended for development + customizability** |
6367
| **Kubernetes** | Production | Yes | |
6468

6569
### Instantiation instructions
@@ -70,7 +74,7 @@ Go to the [WCS console](https://console.weaviate.cloud) and create an instance u
7074

7175
Weaviate instances on WCS come pre-configured, so no further configuration is required.
7276

73-
**Docker-Compose (self-managed)**
77+
**Docker Compose (self-managed)**
7478

7579
Get a configuration file (`docker-compose.yaml`). You can build it using [this interface](https://weaviate.io/developers/weaviate/installation/docker-compose), or download it directly with:
7680

@@ -84,20 +88,20 @@ Where `v<WEAVIATE_VERSION>` is the actual version, such as `v1.18.3`.
8488
curl -o docker-compose.yml "https://configuration.weaviate.io/v2/docker-compose/docker-compose.yml?modules=standalone&runtime=docker-compose&weaviate_version=v1.18.3"
8589
```
8690

87-
**Start up Weaviate with Docker-Compose**
91+
**Start up Weaviate with Docker Compose**
8892

8993
Then you can start up Weaviate by running from a shell:
9094

9195
```shell
92-
docker-compose up -d
96+
docker compose up -d
9397
```
9498

9599
**Shut down Weaviate**
96100

97101
Then you can shut down Weaviate by running from a shell:
98102

99103
```shell
100-
docker-compose down
104+
docker compose down
101105
```
102106

103107
**Notes**
@@ -107,7 +111,7 @@ Unless data persistence or backups are set up, shutting down the Docker instance
107111
See documentation on [Persistent volume](https://weaviate.io/developers/weaviate/installation/docker-compose#persistent-volume) and [Backups](https://weaviate.io/developers/weaviate/configuration/backups) to prevent this if persistence is desired.
108112

109113
```bash
110-
docker-compose up -d
114+
docker compose up -d
111115
```
112116

113117
**Embedded Weaviate (from the application)**
@@ -192,9 +196,7 @@ dbconfig = WeaviateDocumentIndex.DBConfig(
192196
### Create an instance
193197
Let's connect to a local Weaviate service and instantiate a `WeaviateDocumentIndex` instance:
194198
```python
195-
dbconfig = WeaviateDocumentIndex.DBConfig(
196-
host="http://localhost:8080"
197-
)
199+
dbconfig = WeaviateDocumentIndex.DBConfig(host="http://localhost:8080")
198200
doc_index = WeaviateDocumentIndex[MyDoc](db_config=dbconfig)
199201
```
200202

@@ -378,10 +380,10 @@ the [`find()`][docarray.index.abstract.BaseDocIndex.find] method:
378380
embedding=np.array([1, 2]),
379381
file=np.random.rand(100),
380382
)
381-
383+
382384
# find similar documents
383385
matches, scores = doc_index.find(query, limit=5)
384-
386+
385387
print(f"{matches=}")
386388
print(f"{matches.text=}")
387389
print(f"{scores=}")
@@ -428,10 +430,10 @@ You can also search for multiple documents at once, in a batch, using the [`find
428430
)
429431
for i in range(3)
430432
)
431-
433+
432434
# find similar documents
433435
matches, scores = doc_index.find_batched(queries, limit=5)
434-
436+
435437
print(f"{matches=}")
436438
print(f"{matches[0].text=}")
437439
print(f"{scores=}")
@@ -481,7 +483,9 @@ class Book(BaseDoc):
481483
embedding: NdArray[10] = Field(is_embedding=True)
482484

483485

484-
books = DocList[Book]([Book(price=i * 10, embedding=np.random.rand(10)) for i in range(10)])
486+
books = DocList[Book](
487+
[Book(price=i * 10, embedding=np.random.rand(10)) for i in range(10)]
488+
)
485489
book_index = WeaviateDocumentIndex[Book](index_name='tmp_index')
486490
book_index.index(books)
487491

@@ -602,7 +606,7 @@ del doc_index[ids[1:]] # del by list of ids
602606

603607
**WCS instances come pre-configured**, and as such additional settings are not configurable outside of those chosen at creation, such as whether to enable authentication.
604608

605-
For other cases, such as **Docker-Compose deployment**, its settings can be modified through the configuration file, such as the `docker-compose.yaml` file.
609+
For other cases, such as **Docker Compose deployment**, its settings can be modified through the configuration file, such as the `docker-compose.yaml` file.
606610

607611
Some of the more commonly used settings include:
608612

0 commit comments

Comments
 (0)