BigQuery: Add list rows and --max_results option to %%bigquery magic#9147
BigQuery: Add list rows and --max_results option to %%bigquery magic#9147shubha-rajan wants to merge 13 commits intogoogleapis:masterfrom
Conversation
googlebot
added
the
cla: yes
| :type api_response: dict | ||
| :param api_response: response returned from an API call | ||
| Args: | ||
| api_response (dict): response returned from an API call. |
There was a problem hiding this comment.
By the way, the types are specified with names from the typing module, thus a dict should be named Dict, for example. Or Dict[key_type, value_type] if you also want to specify the dict content's type(s). Probably best to check some of the existing "modern" docstrings in the codebase to get a feel.
There was a problem hiding this comment.
got it. The dict in question would be a nested API response so it would be okay to just name it Dict without specifying the content, right?
There was a problem hiding this comment.
I suppose so, yes. If all keys are strings, Dict[str, Any] could be used, but just Dict with a meaningful description is fine, too.
|
failing snippets tests also fail locally on master, so they're probably unrelated to changes in this PR |
|
@shubha-rajan Indeed, that started occurring a day or two ago. The backend team has been informed about it, we are awaiting the ETA for the fix. If it's too long, we can temporarily disable the failing test as a workaround. |
|
Since these are two different features, let's have them as two (possibly 3) separate PRs, starting with I'd prefer we find a different implementation for Let's have 3 PRs in this order:
|
|
@tswast sounds good. I'll close this one and submit 3 separate PRs |
5 participants
See #9105. Creating draft for visibility.
Adds option to pass a table id instead of a SQL query to
%%bigquerycell magic as a cost-saving alternative toSELECT *queries.--max_resultsoption limits the number of rows read. The returnedpandas.DataFramecan be saved to a variable by passing adestination_varargument.TODO:
Fix coverage failures
Test that running cell magic with table ID instead of query works with
bqstorage_clientsetAdd tests for failure cases- handles failure cases when table IDs are passed instead of queries:
max_resultsis currently not working with regular SQL queriesTo get this working, I ended up adding
max_resultsas a property ofQueryJobConfig, but if that wasn't the right call, I can refactor to passmax_resultsas a separate parameter.