Google BigQuery as a target

You can create a data pipeline that loads data from a source endpoint to Google BigQuery. To provide the information SnapLogic AutoSync needs to connect to Google BigQuery, supply new credentials in the wizard or select saved credentials. The create credentials page lists the information necessary to create credentials in the AutoSync wizard.

Supported Account types

Designer and Classic Manager provide multiple account types for most endpoints and not all types are compatible with AutoSync. When you create or edit a data pipeline, the existing credentials list only includes compatible Accounts.

For BigQuery, AutoSync supports:

  • Google BigQuery OAuth2

Known limitations

  • Sometimes AutoSync is unable to clean up the Google Cloud Storage staging area after loading from Salesforce to Google BigQuery. If this occurs, you should manually remove the files to retrieve the storage.
  • Autosync cannot guarantee the replication of source tables with more than 10,000 columns in BigQuery.

Connection configuration

Google BigQuery properties include the following:

  • Credential label: A unique, meaningful name such as >Sales-Shared-BigQuery. If a configuration with the same name exists, AutoSync displays an Asset conflict error message.
  • Authorize: This button authorizes AutoSync with your Google account.
  • Share: (Optional) Select a user group to share this configuration with. Environment admins (formerly Org admins) create user groups to share credentials. If you are a member of a user group, you can select it from the dropdown. You can also select the global shared folder, which shares the credentials with everyone in your Org.
  • Validate and save: After saving and validating, AutoSync adds the configuration to the list of saved credentials.
  • Enter Project ID: Enter the ID of the BigQuery Project to load data into.
  • Select schema: AutoSync populates this list from the account. Choose the schema to use as the destination.
  • Location: (Optional) If you are using a single region schema dataset, you can select the region in the Location field. If you are using a multi-region dataset, you can leave this field blank.
  • Google Cloud Storage bucket name: Enter the name of your Google Cloud Storage bucket to load data into.
  • Log errors in a separate table (one error table for each table): Select to have AutoSync track failed records in an error table.