Load data from other Google and Google Cloud services

You can use a number of Google Cloud services to load data into BigQuery where you can then perform further analysis. These services typically require that you initiate export jobs from the respective console or API of the service. Once enabled, data is loaded into BigQuery according to the cadence defined in the service's export job. Some export jobs run in real time and others provide batch data loads.

For Google Cloud databases and services, including Google Drive and Google Sheets, data queries originate from BigQuery. For more information, see external data sources.

If a service is not listed, you might still be able to export data from the service, but this might require using additional functionality. For more information about how to set up custom exports or how to create load jobs and queries from BigQuery, see Introduction to loading data.

Cloud services that support data exports

Carbon Footprint

The Carbon Footprint export captures gross estimated greenhouse gas emissions associated with the usage of covered Google Cloud services for the selected billing account.

You can export your Carbon Footprint data to BigQuery in order to perform data analysis, or to create custom dashboards and reports.

To set up exports of your Carbon Footprint data, see Export your carbon footprint.

Google Security Operations

You can export Google Security Operations security logs to BigQuery for additional data joins and analytics.

To set up exports of your Google Security Operations security logs, reach out to your Google Security Operations support to set this up.

Cloud Asset Inventory

Cloud Asset Inventory lets you export the asset metadata for your organization, folder, or project to a BigQuery table, and then run data analysis on your inventory.

To set up exports of your Cloud Asset Inventory data, see Exporting to BigQuery.

Cloud Billing

Cloud Billing export to BigQuery lets you export detailed Google Cloud billing data (such as usage, cost estimates, and pricing data) automatically throughout the day.

Timing is important. To have access to a more comprehensive set of billing data for your analysis needs, we recommend that you enable Cloud Billing data export to BigQuery at the same time that you create a Cloud Billing account.

To set up exports of your Cloud Billing data, see Export Cloud Billing data to BigQuery.

Cloud Logging

You can route logs from Cloud Logging to BigQuery tables for additional analytics and joins. For Google Cloud services, log data becomes available for querying approximately 1 minute after it is generated.

To use BigQuery as a part of Log Analytics, see Log Analytics.

To set up exports of your Cloud Logging data, see Route logs to supported sinks.

Conversational Insights

Conversational Insights lets you export your Conversational Insights conversation and analysis data to BigQuery so that you can perform your own raw queries.

To configure exports of your Conversational Insights data, see Export conversations to BigQuery.

Dialogflow CX

Dialogflow CX generates logs of the conversations between agents and your customers.

To configure exports of conversations from Dialogflow CX, see Interaction logging export to BigQuery.

Firebase

Firebase contains a number of analytics exports you can send to BigQuery. These include:

  • Analytics
  • Cloud messaging
  • Crashlytics
  • Performance monitoring
  • A/B testing
  • Remote configuration personalization

To configure exports of Firebase data, see Export project data to BigQuery.

Google Analytics 4

To learn how to export your session data from a Google Analytics 4 reporting view into BigQuery, see BigQuery export Set up BigQuery export in the Analytics Help Center. After the Google Analytics 4 data is in BigQuery, you can query it by using GoogleSQL.

If you are experiencing errors with your Google Analytics 4 export, see Reasons for linking failures or Reasons for export failures.

If you are experiencing data delays from Google Analytics 4 exports, missing Google Analytics 4 data after an export, or other issues, contact Analytics support.

For BigQuery issues, like billing, contact Google Cloud Support.

Google Analytics 360

To learn how to export your session data from a Google Analytics 360 reporting view into BigQuery, see BigQuery export in the Analytics Help Center. After the Google Analytics 360 data is in BigQuery, you can query it by using GoogleSQL.

For examples of querying Analytics data in BigQuery, see BigQuery cookbook in the Analytics Help.

For issues related to linking BigQuery and Google Analytics 360, missing Google Analytics 360 data after an export, or issues related to data delays when exporting from Google Analytics 360, contact Google Analytics 360 support.

For BigQuery issues, like billing, contact Google Cloud Support.

You can schedule a daily export of your Google Search Console performance data to BigQuery, where you can run complex queries over your data.

To set up exports of your data, see About bulk data export of Search Console data to BigQuery.

Recommender

You can schedule daily snapshots of the recommendations using the BigQuery Data Transfer Service. Recommendations provide advice on optimizing your usage of Google Cloud products and resources, and also provide insights into your resource usage patterns.

To set up snapshots of your data using the BigQuery Data Transfer Service, see Export recommendations to BigQuery.

Vertex AI Batch Prediction

Vertex AI Batch Prediction creates a set of predictions based on an input to a model. You can store these results in BigQuery for additional analytics and joins.

To configure exports of batch prediction results, see Get batch predictions and explanations.

Vertex AI Predictions

You can use Vertex AI Predictions to store prediction results from online endpoints in BigQuery for additional analysis.

To configure model prediction integration with BigQuery, see Online prediction logging.

What's next