Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: query_and_wait now retains unknown query configuration _properties #1793

Merged
merged 4 commits into from
Jan 23, 2024

Conversation

tswast
Copy link
Contributor

@tswast tswast commented Jan 19, 2024

fix: raise ValueError in query_and_wait with wrong job_config type

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #1792
🦕

@tswast tswast requested review from a team as code owners January 19, 2024 17:20
@tswast tswast requested a review from Neenu1995 January 19, 2024 17:20
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery API. labels Jan 19, 2024
@tswast tswast changed the title fix: query_and_wait now retains unknown query configuration `_prope… fix: query_and_wait now retains unknown query configuration _properties Jan 19, 2024
@tswast tswast requested a review from Linchin January 19, 2024 17:20
@tswast tswast enabled auto-merge (squash) January 19, 2024 20:26
@Linchin
Copy link
Contributor

Linchin commented Jan 19, 2024

Could you tell me more about the rationale of keeping unknown config fields?

@tswast
Copy link
Contributor Author

tswast commented Jan 22, 2024

Could you tell me more about the rationale of keeping unknown config fields?

Two reasons: (1) as you can see from the removal of this code:

    # These keys are top level in job resource and query resource.
    if "labels" in job_config_resource:
        request_body["labels"] = job_config_resource["labels"]
    if "dryRun" in job_config_resource:
        request_body["dryRun"] = job_config_resource["dryRun"]

it allows for a cleaner conversion from query job insert request to jobs.query request.

(2) It allows for customers who want access to features that are in preview an not yet implemented in the client library to set REST API properties via setting QueryJobConfig._properties. This is a very common practice (see: #1398 (comment) for example) and is in fact a requirement for features that are in a "private" release. We don't support adding private features to our public client library, but this _properties feature allows such customers to use the public client library via this workaround.

@Linchin
Copy link
Contributor

Linchin commented Jan 23, 2024

That's great to know! Thank you

@tswast tswast merged commit 4ba4342 into main Jan 23, 2024
21 of 22 checks passed
@tswast tswast deleted the issue1792-query_and_wait-invalid-job_config branch January 23, 2024 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: query_and_wait drops unknown properties in the QueryJobConfig
3 participants