chore(docs): support pydantic data model

hanxiao · hanxiao · commit b3debdeacb65 · 2022-01-14T21:54:15.000+01:00
diff --git a/README.md b/README.md
@@ -14,13 +14,13 @@
 
 DocArray is a library for nested, unstructured data such as text, image, audio, video, or 3D mesh. It allows deep-learning engineers to efficiently process, embed, search, recommend, store, and transfer the data with a Pythonic API.
 
-&#127756; **All data types**: super-expressive data structure for representing complicated/mixed/nested text, image, video, audio, 3D mesh data.
+&#127756; **Rich data types**: super-expressive data structure for representing complicated/mixed/nested text, image, video, audio, 3D mesh data.
 
 &#128013; **Pythonic experience**: designed to be as easy as a Python list. If you know how to Python, you know how to DocArray. Intuitive idioms and type annotation simplify the code you write.
 
 &#129489;&zwj;&#128300; **Data science powerhouse**: greatly accelerate data scientists' work on embedding, matching, visualizing, evaluating via Torch/TensorFlow/ONNX/PaddlePaddle on CPU/GPU.
 
-&#128673; **Portable**: ready-to-wire at anytime with efficient and compact serialization from/to Protobuf, bytes, base64, JSON, CSV, DataFrame.
+&#128673; **Portable**: ready-to-wire at anytime with fast and compressed serialization from/to Protobuf, bytes, base64, JSON, CSV, DataFrame. Built-in data validation and JSON Schema (OpenAPI) help you build reliable webservices.
 
 <!-- end elevator-pitch -->
 
diff --git a/docs/fundamentals/document/serialization.md b/docs/fundamentals/document/serialization.md
@@ -10,10 +10,16 @@ One should use {ref}`DocumentArray for serializing multiple Documents<docarray-s
 
 ## From/to JSON
 
+```{tip}
+If you are building a webservice and want to use JSON for passing DocArray objects, then data validation and field-filtering can be crucial. In this case, it is highly recommended to check out {ref}`fastapi-support` and follow the methods there.   
+```
+
 ```{important}
 This feature requires `protobuf` dependency. You can do `pip install "docarray[full]"` to install it.
 ```
 
+
+
 You can serialize a Document as a JSON string via {meth}`~docarray.document.mixins.porting.PortingMixin.to_json`, and then read from it via {meth}`~docarray.document.mixins.porting.PortingMixin.from_json`.
 
 ```python
diff --git a/docs/fundamentals/documentarray/serialization.md b/docs/fundamentals/documentarray/serialization.md
@@ -3,7 +3,8 @@
 
 DocArray is designed to be "ready-to-wire" at anytime. Serialization is important. DocumentArray provides multiple serialization methods that allows one transfer DocumentArray object over network and across different microservices.
 
-- JSON string: `.from_json()`/`.to_json()` 
+- JSON string: `.from_json()`/`.to_json()`
+  - Pydantic model: `.from_pydantic_model()`/`.to_pydantic_model()`
 - Bytes (compressed): `.from_bytes()`/`.to_bytes()`
 - Base64 (compressed): `.from_base64()`/`.to_base64()` 
 - Protobuf Message: `.from_protobuf()`/`.to_protobuf()`
@@ -13,10 +14,17 @@ DocArray is designed to be "ready-to-wire" at anytime. Serialization is importan
 
 ## From/to JSON
 
+
+```{tip}
+If you are building a webservice and want to use JSON for passing DocArray objects, then data validation and field-filtering can be crucial. In this case, it is highly recommended to check out {ref}`fastapi-support` and follow the methods there.   
+```
+
 ```{important}
 This feature requires `protobuf` dependency. You can do `pip install "docarray[full]"` to install it.
 ```
 
+
+
 ```python
 from docarray import DocumentArray, Document
 
diff --git a/docs/fundamentals/fastapi-support/index.md b/docs/fundamentals/fastapi-support/index.md
@@ -1,3 +1,4 @@
+(fastapi-support)=
 # FastAPI/pydantic Support
 
 Long story short, DocArray supports [pydantic data model](https://pydantic-docs.helpmanual.io/) via {class}`~docarray.document.pydantic_model.PydanticDocument` and {class}`~docarray.document.pydantic_model.PydanticDocumentArray`.
diff --git a/docs/get-started/what-is.md b/docs/get-started/what-is.md
@@ -37,6 +37,7 @@ DocArray is designed to maximize the local experience, with the requirement of c
 | Nested data                     |&#9989;|&#10060;| &#9989;    |&#10060;|&#9989;|
 | Mixed data of the above four    |&#9989;|&#10060;| &#10060;    |&#10060;|&#10060;|
 | Easy to (de)serialize           |&#9989;|&#10060;| &#9989;    |&#9989;|&#9989;|
+| Data validation (of the output) |&#9989;|&#10060;| &#10060;    |&#10060;|&#9989;|
 | Pythonic experience             |&#9989;|&#9989;| &#10060;    |&#10004;&#65039;&#65039;|&#10060;|
 | IO support for filetypes        |&#9989;|&#10060;| &#10060;    |&#10060;|&#10060;|
 | Deep learning framework support |&#9989;|&#9989;| &#10060;    |&#10060;|&#10060;|
@@ -118,6 +119,8 @@ Beside code refactoring and optimization, many features have been improved, incl
 - revised documentations and examples
 - ... and many more.
 
+When first using DocArray, some Jina 2.x user may realize the static typing seems missing. This is due to a deliberate decision of DocArray: DocArray guarantees the types and constraints of the wire data, not the input data. In other words, only the functions that are listed under {ref}`docarray-serialization` chapter will trigger the data validation. 
+
 To learn DocArray, the recommendation here is to forget about everything in Jina 2.x, although some interfaces may look familiar. Read [the fundamental sections](../fundamentals/document/index.md) from beginning.
 
 ```{important}

-Original file line number
+Diff line change
 DocArray is a library for nested, unstructured data such as text, image, audio, video, or 3D mesh. It allows deep-learning engineers to efficiently process, embed, search, recommend, store, and transfer the data with a Pythonic API.
 -🌌 **All data types**: super-expressive data structure for representing complicated/mixed/nested text, image, video, audio, 3D mesh data.
 +🌌 **Rich data types**: super-expressive data structure for representing complicated/mixed/nested text, image, video, audio, 3D mesh data.
 🐍 **Pythonic experience**: designed to be as easy as a Python list. If you know how to Python, you know how to DocArray. Intuitive idioms and type annotation simplify the code you write.
 🧑‍🔬 **Data science powerhouse**: greatly accelerate data scientists' work on embedding, matching, visualizing, evaluating via Torch/TensorFlow/ONNX/PaddlePaddle on CPU/GPU.
 -🚡 **Portable**: ready-to-wire at anytime with efficient and compact serialization from/to Protobuf, bytes, base64, JSON, CSV, DataFrame.
 +🚡 **Portable**: ready-to-wire at anytime with fast and compressed serialization from/to Protobuf, bytes, base64, JSON, CSV, DataFrame. Built-in data validation and JSON Schema (OpenAPI) help you build reliable webservices.
 <!-- end elevator-pitch -->
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,4 @@`
	`1`	`+(fastapi-support)=`
`1`	`2`	`# FastAPI/pydantic Support`
`2`	`3`
`3`	`4`	Long story short, DocArray supports [pydantic data model](https://pydantic-docs.helpmanual.io/) via {class}`~docarray.document.pydantic_model.PydanticDocument` and {class}`~docarray.document.pydantic_model.PydanticDocumentArray`.