Skip to content

feat(v2): rich display for doc and da #1043

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
samsja merged 46 commits into from
Jan 25, 2023
Merged

feat(v2): rich display for doc and da #1043

Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
6e9817c
feat: add rich display for doc and da
Jan 17, 2023
2941269
fix: wip plot
Jan 18, 2023
5949e7c
fix: wip plot
Jan 18, 2023
05fa0fa
fix: wip plot
Jan 19, 2023
718fe52
feat: add math package and minmax normalize
Jan 19, 2023
3669de1
fix: summary for document
Jan 19, 2023
c56e975
chore: update poetry lock after rebase
Jan 19, 2023
b0ba3f3
fix: move all from plotmixin to base document
Jan 19, 2023
bd8cf3b
feat: add docs schema summary
Jan 20, 2023
25be9cc
feat: add document array summary
Jan 20, 2023
b7a915b
fix: display doc within doc
Jan 20, 2023
c6ee8ec
fix: in notebook print docs summary
Jan 20, 2023
d45988a
fix: move summary from da to abstract da
Jan 23, 2023
40c8eea
fix: get schema for doc
Jan 23, 2023
3bdb9d0
fix: wip doc summary
Jan 23, 2023
ea12600
fix: wip clean up
Jan 23, 2023
9321c0b
test: add test for da pretty print
Jan 23, 2023
189c33c
docs: update note
Jan 23, 2023
93046af
docs: add some documentation
Jan 23, 2023
fc0deec
fix: apply samis suggestion
Jan 23, 2023
c8f3849
fix: mypy checks
Jan 23, 2023
15b94fc
fix: move to plot mixin
Jan 23, 2023
58229aa
fix: remove redundant line
Jan 24, 2023
e55ba3b
fix: remove comments
Jan 24, 2023
147742d
feat: add schema highlighter
Jan 24, 2023
59bd3a6
fix: add plotmixin to mixin init
Jan 24, 2023
fd26a43
fix: adjust da summary
Jan 24, 2023
675b5c5
fix: move minmaxnormalize to comp backend
Jan 24, 2023
a375d19
fix: remove redundant lines
Jan 24, 2023
c3b44bd
fix: add squeeze and detach to comp backend
Jan 24, 2023
0d5653c
fix: apply suggestion from code review
Jan 24, 2023
6d479ab
refactor: rename iterable attrs
Jan 24, 2023
a1c4678
fix: clean up
Jan 24, 2023
3aac1c9
fix: import
Jan 24, 2023
eb75060
fix: iterate over fields instead of annotations
Jan 24, 2023
3cc1b55
fix: remove math package since moved to comp backends
Jan 24, 2023
ab585eb
refactor: use single quotes
Jan 24, 2023
b838ec9
fix: apply suggestions from code review
Jan 24, 2023
c56aa6e
fix: extract summary to doc summary class
Jan 24, 2023
b2b5bdd
fix: add pretty print for base document
Jan 25, 2023
7aa7e58
fix: use rich capture instead of string io
Jan 25, 2023
2ae8d6a
fix: add colors for optional and union and use only single quotes
Jan 25, 2023
0b881b1
fix: extract display classes to display package
Jan 25, 2023
6ba4eff
fix: make da not optional in da summary
Jan 25, 2023
a70142a
fix: set _console instead of initializing new one everytime in __str__
Jan 25, 2023
2a6bd5c
fix: put console at module level
Jan 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: use rich capture instead of string io 
Signed-off-by: anna-charlotte <[email protected]>
  • Loading branch information
anna-charlotte committed Jan 25, 2023
commit 7aa7e586b8b29b91457ead1143c46ef0a6805bed
9 changes: 5 additions & 4 deletions docarray/base_document/document.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import os
from io import StringIO
from typing import Type

import orjson
Expand Down Expand Up @@ -38,6 +37,8 @@ def _get_field_type(cls, field: str) -> Type['BaseDocument']:
return cls.__fields__[field].outer_type_

def __str__(self):
output = StringIO()
rich.print(self, file=output)
return output.getvalue().strip()
console = rich.console.Console()
Copy link
Member

@samsja samsja Jan 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to create a console each time or can we define it once ?

Copy link
Contributor Author

@anna-charlotte anna-charlotte Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, if I set it to self.console outside of def __str__() I am getting this error:

Traceback (most recent call last):
  File "/Users/charlottegerhaher/Library/Application Support/JetBrains/PyCharm2022.2/scratches/scratch_32.py", line 6, in <module>
    from docarray import BaseDocument, DocumentArray
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/__init__.py", line 3, in <module>
    from docarray.array.array import DocumentArray
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/array/__init__.py", line 1, in <module>
    from docarray.array.array import DocumentArray
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/array/array.py", line 7, in <module>
    from docarray.array.abstract_array import AnyDocumentArray
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/array/abstract_array.py", line 4, in <module>
    from docarray.base_document import BaseDocument
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/base_document/__init__.py", line 1, in <module>
    from docarray.base_document.any_document import AnyDocument
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/base_document/any_document.py", line 3, in <module>
    from .document import BaseDocument
  File "/Users/charlottegerhaher/Desktop/jina-ai/docarray_v2/docarray/docarray/base_document/document.py", line 15, in <module>
    class BaseDocument(BaseModel, PlotMixin, ProtoMixin, AbstractDocument, BaseNode):
  File "pydantic/main.py", line 198, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 546, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 570, in pydantic.fields.ModelField._set_default_and_type
  File "pydantic/fields.py", line 439, in pydantic.fields.ModelField.get_default
  File "pydantic/utils.py", line 695, in pydantic.utils.smart_deepcopy
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle '_thread.RLock' object

Copy link
Contributor Author

@anna-charlotte anna-charlotte Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah wait, my bad, this works, too. I put it as BaseDocuments attribute for now:

class BaseDocument(BaseModel, PlotMixin, ProtoMixin, AbstractDocument, BaseNode):
    """
    The base class for Document
    """
    id: ID = Field(default_factory=lambda: parse_obj_as(ID, os.urandom(16).hex()))
    _console: rich.console.Console = rich.console.Console()
    ...

We could even put it outside of the class and use the same console object for all Documents, instead of one for each:

_console: rich.console.Console = rich.console.Console()


class BaseDocument(BaseModel, PlotMixin, ProtoMixin, AbstractDocument, BaseNode):
    """
    The base class for Document
    """
    ...

Copy link
Member

@samsja samsja Jan 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I like the later better only one console object at the module level

with console.capture() as capture:
console.print(self)

return capture.get().strip()