BigQuery Table Data List

Use this Snap to read and retrieve data from a Google BigQuery table.

Overview

You can use this Snap to read and retrieve data from a Google BigQuery table.



Prerequisites

A valid Google BigQuery account with the required permissions.

Limitations and Known Issues

None.

Supported Accounts

This Snap works with the following account types. For more information, see Configuring Google BigQuery Accounts.

Snap Views

Type Format Number of Views Examples of Upstream and Downstream Snaps Description
Input Document
  • Min: 0
  • Max: 1
  • Mapper
  • JSON Generator

The Project ID and the Document ID.
Output Document
  • Min: 1
  • Max: 1
  • JSON Parser
  • File Writer

The list of Table IDs along with their Project IDs, Dataset IDs, and Table type.
Learn more about Error handling.

Snap Settings

Note: Learn about the common controls in the Snap settings dialog.
Field Name/Type Field Dependency Description
Label

String/Expression

 None.

Enter a unique name for the Snap. You can modify this to be more specific, especially if you have more than one of the same Snap in your pipeline.

Default value: BigQuery Table Data List

Example: Sales Leads Data List
Project ID

String/Expression

 None.

Specify the project ID in which the dataset resides.

Default value: N/A

Example: test-project-12345
Dataset ID

String/Expression

 None.

Specify the dataset ID of the destination.

Default value: N/A

Example: dataset-12345
Table ID

String/Expression

 None.

Specify the table ID of the table you are creating.

Default value: N/A

Example: table-12345
Snap execution None.
Choose one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
  • Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default value: Validate & Execute

Example: Disabled

Implicit retries in BigQuery Snaps

The BigQuery Snaps handle all retriable BigQuery errors (BigQuery exception, IO exception, and Runtime exception) internally.

  • 429 (Too Many Requests):
    • Retry attempts: Maximum of 5 retries.
    • Delay Between Retries: Backoff strategy with jitter (random variation) is applied to prevent synchronized retries and reduce load.
  • 401 (Unauthorized):
    • Retry attempts: Maximum of 3 retries.
    • Delay Between Retries: Backoff strategy is applied.
    • Additional Actions: Reloads the BigQuery account on the retry event.
  • IOException and 500, 502, 503, 504 (Server Errors):
    • Retry attempts: Maximum of 3 retries.
    • Delay Between Retries: Backoff strategy is applied.