Snowflake - Multi Execute

Overview

You can use this Snap to execute multiple queries. If any query fails in execution, the changes are rolled back.

Note:

The Snowflake - Multi Execute Snap supports executing all Snowflake queries and does only write operations.


Snowflake Multi Execute Configuration

Prerequisites

Read and write access to the Snowflake instance.

You must have minimum permissions on the database to execute Snowflake Snaps. To understand if you already have them, you must retrieve the current set of permissions. The following commands enable you to retrieve those permissions: 

SHOW GRANTS ON DATABASE <database_name>
        SHOW GRANTS ON SCHEMA <schema_name>
        SHOW GRANTS TO USER <user_name>
Security Prererequisites
You must have the following permissions in your Snowflake account to execute this Snap:
  • Usage (DB and Schema): Privilege to use the database, role, and schema.
  • Create table: Privilege to create a temporary table within this schema.
The following commands enable minimum privileges in the Snowflake Console:
grant usage on database <database_name> to role <role_name>;
              grant usage on schema <database_name>.<schema_name>;
              
              grant "CREATE TABLE" on database <database_name> to role <role_name>;
              grant "CREATE TABLE" on schema <database_name>.<schema_name>;

Learn more about Snowflake privileges: Access Control Privileges.

Internal SQL Commands

The permissions to grant for usage on database and creating tables depend on the queries you provide in this Snap.

Limitations

  • The Snap may fail due to an error when the batch size specified in the account is 1 and the number of rows updated by an executed query exceeds 2.1 billion.
  • Because of performance issues, all Snowflake Snaps now ignore the Cancel queued queries when pipeline is stopped or if it fails option for Manage Queued Queries, even when selected. Snaps behave as though the default Continue to execute queued queries when the Pipeline is stopped or if it fails option were selected.
  • Select statement and transactional statements such as Begin, Commit, and Rollback are not supported.
  • Changes made to the database using Data Definition Language (DDL) queries are not rolled back if the Snap fails, this is since DDL statements are always auto-committed by default. If there is one DDL query among other queries that the Snap is executing, then the rollback is only for the changes made by the queries executed after the DDL query. Ensure that the Disable session auto-commit property is selected when using DDL queries. See Snowflake Auto-commit for details. This is not applicable to Data Modification Language (DML) queries.
  • User-defined functions (UDFs) created in the Snowflake console can be executed using Snowflake - Multi Execute Snap. See Snowflake Multi Execute-Examples.

Snap views

View Description Examples of upstream and downstream Snaps
Input While the Snap does not require an input document, it supports document input and generates document outputs. Snaps that accept or generate documents can be used upstream or downstream of the Snap.
Output The Snap will output one document for every record retrieved, hence any document processing Snap can be used downstream.
Error

Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:

  • Stop Pipeline Execution Stops the current pipeline execution when an error occurs.
  • Discard Error Data and Continue Ignores the error, discards that record, and continues with the remaining records.
  • Route Error Data to Error View Routes the error data to an error view without stopping the Snap execution.

Learn more about Error handling in Pipelines.

Snap settings

Legend:
  • Expression icon (): Allows using JavaScript syntax to access SnapLogic Expressions to set field values dynamically (if enabled). If disabled, you can provide a static value. Learn more.
  • SnapGPT (): Generates SnapLogic Expressions based on natural language using SnapGPT. Learn more.
  • Suggestion icon (): Populates a list of values dynamically based on your Snap configuration. You can select only one attribute at a time using the icon. Type into the field if it supports a comma-separated list of values.
  • Upload : Uploads files. Learn more.
Learn more about the icons in the Snap settings dialog.
Field / Field set Type Description
Label String Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if there are more than one of the same Snap in the pipeline.
SQL Statements String/Expression Required. Specify the Snowflake SQL statement to execute on the server. Provide the queries to be executed by the Snap. End each query with a semi-colon, except the last query. To add another query, click the + to add a row and there add the query.
Note: Using Where Clauses

Valid JSON paths that are defined in the WHERE clause for queries/statements are substituted with values from an incoming document. If the error view is enabled, documents are written to the error view if the document is missing a value to be substituted into the query/statement.

Without using expressionsUsing expressions
  • "EMPNO=$EMPNO and ENAME=$EMPNAME"
  • email = $email
  • emp=$emp
  • "emp='" + $emp + "'"
  • "EMPNO=" + $EMPNO + " and ENAME='" + $EMPNAME+ "'"
Note: Using expressions that join strings together to create SQL queries or conditions has a potential SQL injection risk and is hence unsafe. Ensure that you understand all implications and risks involved before using concatenation of strings with '=' Expression enabled.
Note: Single quotes in values must be escaped

Any relational database (RDBMS) treats single quotes (') as special symbols. So, single quotes in the data or values passed through a DML query may cause the Snap to fail when the query is executed. Ensure that you pass two consecutive single quotes in place of one within these values to escape the single quote through these queries.

If String To pass this value Use
Has no single quotes Schaum Series 'Schaum Series'
Contains single quotes O'Reilly's Publication 'O''Reilly''s Publication'

Default value: N/A

Example: INSERT into SnapLogic.book (id, book) VALUES ($id,$book)
Session auto-commit Checkbox Select this to enable the session's auto-commit. This is useful when using DDL queries since the changes made by such queries cannot be rolled back if the Snap/pipeline fails. If deselected, changes made by DML queries can be rolled back or changes made by DML queries after a DDL queries can be rolled back.

Default status: Selected
Use Result Query Checkbox Select this checkbox to write the query execution result to the Snap's output view after the successful execution. The output of the Snap will be enclosed within the key Result Query, and the value will be the actual output produced by the SQL query.

This option allows users to effectively track the query's execution by clearly indicating the successful execution and the number of records affected, if any, after the execution.

Default status: deselected
Manage Queued Queries Dropdown list
Default value: Select this property to determine whether the Snap should continue or cancel the execution of the queued Snowflake Execute SQL queries when you stop the pipeline.
Note: If you select Cancel queued queries when the pipeline is stopped or if it fails, then the read queries under execution are canceled, whereas the write type of queries under execution are not canceled. Snowflake internally determines which queries are safe to be canceled and cancels those queries.

Default value: Continue to execute queued queries when the Pipeline is stopped or if it fails

Example: Cancel queued queries when the pipeline is stopped or if it fails
Snap execution Dropdown list
Choose one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
  • Execute only: Performs full execution of the Snap during pipeline execution without generating preview data.
  • Disabled: Disables the Snap and all Snaps that are downstream from it.

Default value: Execute only

Example: Validate & Execute

Troubleshooting

Unable to set auto-commit for this connection (Session ID: <Session ID number>)

Manually terminate the session via the Snowflake console. Learn more: How to terminate a session in Snowflake.

Data can only be read from Google Cloud Storage (GCS) with the supplied account credentials (not written to it).

Snowflake Google Storage Database accounts do not support external staging when the Data source is the Input view.

Data can only be read from GCS with the supplied account credentials (not written to it).

Use internal staging if the data source is the input view or change the data source to staged files for Google Storage external staging.