Skip to content

Infer schema of PyArrow table directly#355

Merged
feast-ci-bot merged 1 commit intomasterfrom
pyarrow-type-dev
Dec 22, 2019
Merged

Infer schema of PyArrow table directly#355
feast-ci-bot merged 1 commit intomasterfrom
pyarrow-type-dev

Conversation

@voonhous
Copy link
Copy Markdown
Collaborator

@voonhous voonhous commented Dec 9, 2019

  • Added support to infer schema of PyArrow table directly (All types supported except bool_list_val)
  • Removed schema inference with intermediate pandas DataFrame
  • Added confluent-kafka support to improve throughput of FeatureRows being pushed into Kafka
  • Added a series of optimizations to reduce memory footprint

@voonhous
Copy link
Copy Markdown
Collaborator Author

voonhous commented Dec 9, 2019

/retest

1 similar comment
@voonhous
Copy link
Copy Markdown
Collaborator Author

voonhous commented Dec 9, 2019

/retest

@voonhous
Copy link
Copy Markdown
Collaborator Author

/retest

@voonhous
Copy link
Copy Markdown
Collaborator Author

/retest

@woop
Copy link
Copy Markdown
Member

woop commented Dec 22, 2019

/lgtm

@woop
Copy link
Copy Markdown
Member

woop commented Dec 22, 2019

/approved

@feast-ci-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: voonhous, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@feast-ci-bot feast-ci-bot merged commit 437b350 into master Dec 22, 2019
@feast-ci-bot
Copy link
Copy Markdown
Collaborator

@voonhous: Updated the config configmap in namespace default at cluster default using the following files:

  • key config.yaml using file .prow/config.yaml
Details

In response to this:

  • Added support to infer schema of PyArrow table directly (All types supported except bool_list_val)
  • Removed schema inference with intermediate pandas DataFrame
  • Added confluent-kafka support to improve throughput of FeatureRows being pushed into Kafka
  • Added a series of optimizations to reduce memory footprint

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@voonhous voonhous deleted the pyarrow-type-dev branch December 22, 2019 15:47
davidheryanto pushed a commit that referenced this pull request Jan 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants