feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob #1521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…QueryJob Note for google-cloud-bigquery developers: This also refactors these classes so that `_set_properties` does not modify the `_properties` dictionary in-place. Doing so was also mutating the request object, making it difficult to debug what request was _actually_ sent. Before this change, many tests hallucinated that the request was always equal to the response.
product-auto-label
bot
added
size: l
tswast
commented
Mar 10, 2023
| @@ -1976,15 +1976,8 @@ def create_job( | |||
| ) | |||
| destination = _get_sub_prop(job_config, ["copy", "destinationTable"]) | |||
| destination = TableReference.from_api_repr(destination) | |||
| sources = [] | |||
There was a problem hiding this comment.
There was a unit test failing because we were actually transforming "sourceTable" to "sourceTables" without this change.
…te disposition WRITE_TRUNCATE is not supported. Please try again with WRITE_EMPTY.
|
P.S. Thanks @chelsea-lin for identifying this bug! |
This was referenced Apr 4, 2023
chalmerlowe
pushed a commit
that referenced
this pull request
Apr 13, 2023
…QueryJob (#1521) * feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob Note for google-cloud-bigquery developers: This also refactors these classes so that `_set_properties` does not modify the `_properties` dictionary in-place. Doing so was also mutating the request object, making it difficult to debug what request was _actually_ sent. Before this change, many tests hallucinated that the request was always equal to the response. * E google.api_core.exceptions.BadRequest: 400 Clone operation with write disposition WRITE_TRUNCATE is not supported. Please try again with WRITE_EMPTY.
This was referenced Apr 17, 2023
abdelmegahedgoogle
pushed a commit
to abdelmegahedgoogle/python-bigquery
that referenced
this pull request
Apr 17, 2023
…QueryJob (googleapis#1521) * feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob Note for google-cloud-bigquery developers: This also refactors these classes so that `_set_properties` does not modify the `_properties` dictionary in-place. Doing so was also mutating the request object, making it difficult to debug what request was _actually_ sent. Before this change, many tests hallucinated that the request was always equal to the response. * E google.api_core.exceptions.BadRequest: 400 Clone operation with write disposition WRITE_TRUNCATE is not supported. Please try again with WRITE_EMPTY.
chalmerlowe
added a commit
that referenced
this pull request
Apr 18, 2023
#1541) * bug: fixes discrepancy btwn python-api-core & bigquery re object default timeout * Fix: loosen ipywidget dependency (#1504) * fix: updates ipywidget dependency * fix: updates ipywidget version number * chore(main): release 3.6.0 (#1490) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * docs: Remove < 3.11 reference from README (#1502) * chore(python): upgrade gcp-releasetool in .kokoro [autoapprove] (#1508) Source-Link: googleapis/synthtool@5f2a608 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:8555f0e37e6261408f792bfd6635102d2da5ad73f8f09bcb24f25e6afb5fac97 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: add `connection_properties` and `create_session` to `LoadJobConfig` (#1509) * feat: added `connection_properties` and `create_session` in load job * chore(deps): update all dependencies (#1501) * chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Anthonios Partheniou <[email protected]> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * chore(deps): update all dependencies (#1513) * feat: add default_query_job_config property and property setter to BQ client (#1511) Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly: - [x] Make sure to open an issue as a [feature request](https://togithub.com/googleapis/python-bigquery/issues/1512) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea - [x] Ensure the tests and linter pass - [x] Code coverage does not decrease (if any source code was changed) - [x] Appropriate docs were updated (if necessary) Fixes - [feature request](https://togithub.com/googleapis/python-bigquery/issues/1512)🦕 - [internal bug](https://b.corp.google.com/issues/271044948) * chore(deps): update all dependencies (#1514) * chore(deps): update dependency charset-normalizer to v3.1.0 (#1518) * chore(main): release 3.7.0 (#1507) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob (#1521) * feat: expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob Note for google-cloud-bigquery developers: This also refactors these classes so that `_set_properties` does not modify the `_properties` dictionary in-place. Doing so was also mutating the request object, making it difficult to debug what request was _actually_ sent. Before this change, many tests hallucinated that the request was always equal to the response. * E google.api_core.exceptions.BadRequest: 400 Clone operation with write disposition WRITE_TRUNCATE is not supported. Please try again with WRITE_EMPTY. * chore(deps): Update nox in .kokoro/requirements.in [autoapprove] (#1527) Source-Link: googleapis/synthtool@92006bb Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:2e247c7bf5154df7f98cce087a20ca7605e236340c7d6d1a14447e5c06791bd6 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: add default LoadJobConfig to Client (#1526) * feat: add bool, int, float, string dtype to to_dataframe (#1529) * fix: loosen ipywidgets restrictions further to address ipython compatibility issues (#1531) * fix: loosen ipywidgets restrictions further to address ipython compatibility issues * include ipywidgets in prerelease deps * show all package versions * add ipykernel dependency * ipykernel in noxfile * oops * chore(main): release 3.8.0 (#1525) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * fix: keyerror when the load_table_from_dataframe accesses a unmapped dtype dataframe index (#1535) * feat: expose query job on dbapi cursor (#1520) Co-authored-by: Tim Swast <[email protected]> * chore(main): release 3.9.0 (#1537) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * chore: update tests to be compatible with pandas 2.0 (#1538) * chore: update tests to be compatible with pandas 2.0 * use StringDtype without storage argument * avoid Float64Dtype on older pandas * chore(deps): update all dependencies (#1522) * chore: updates minimum version of bqstorage (#1542) * chore: updates minimum version of bqstorage * removes unneeded test * updates linting, removes unneeded comment * updates conditional checks, comments, adds test * Removes test, adds pragma no cover * Removes test * fix linting error --------- Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Atsushi Yamamoto <[email protected]> Co-authored-by: gcf-owl-bot[bot] <78513119+gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Shobhit Singh <[email protected]> Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Anthonios Partheniou <[email protected]> Co-authored-by: chelsea-lin <[email protected]> Co-authored-by: Tim Swast <[email protected]> Co-authored-by: r1b <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note for google-cloud-bigquery developers: This also refactors these classes so that
_set_propertiesdoes not modify the_propertiesdictionary in-place. Doing so was also mutating the request object, making it difficult to debug what request was actually sent. Before this change, many tests hallucinated that the request was always equal to the response.Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
🦕