BigQuery Dataset List

Use this Snap to list BigQuery Datasets.

Overview

You can use this Snap to list BigQuery Datasets.



Prerequisites

A valid Google BigQuery Account with the required permissions.

Limitations and Known Issues

None.

Supported Accounts

Snap Views

Type Format Number of Views Examples of Upstream and Downstream Snaps Description
Input Document
  • Min: 0
  • Max: 1
  • Mapper
  • JSON Generator

The Project ID from which the Snap reads the dataset to be listed.
Output Document
  • Min: 1
  • Max: 1
  • File Writer
  • Mapper

The dataset list.
Learn more about Error handling.

Snap Settings

Note: Learn about the common controls in the Snap settings dialog.
Field/Field set Description
Label

String

Specify a unique name for the Snap.

Default value: BigQuery Dataset List

Example: List Emp Dataset
Project ID

String/Expression

Specify the Project ID from which the dataset is to be listed.

Default value: N/A

Example: Test-project-1234
Detailed Information

Checkbox

Select this checkbox to enable the Snap to access additional fields for displaying them in the output.

Default value: Deselected
Snap execution

Dropdown list
Choose one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute. Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
  • Execute only. Performs full execution of the Snap during pipeline execution without generating preview data.
  • Disabled. Disables the Snap and all Snaps that are downstream from it.

Default value: Execute only

Example: Validate & Execute

Implicit retries in BigQuery Snaps

The BigQuery Snaps handle all retriable BigQuery errors (BigQuery exception, IO exception, and Runtime exception) internally.

  • 429 (Too Many Requests):
    • Retry attempts: Maximum of 5 retries.
    • Delay Between Retries: Backoff strategy with jitter (random variation) is applied to prevent synchronized retries and reduce load.
  • 401 (Unauthorized):
    • Retry attempts: Maximum of 3 retries.
    • Delay Between Retries: Backoff strategy is applied.
    • Additional Actions: Reloads the BigQuery account on the retry event.
  • IOException and 500, 502, 503, 504 (Server Errors):
    • Retry attempts: Maximum of 3 retries.
    • Delay Between Retries: Backoff strategy is applied.