Salesforce Bulk Query

Overview

The Salesforce Bulk Query Snap performs SOQL queries using Salesforce Bulk API 2.0.

Note: To use newer Salesforce objects, use the latest API versions where those objects are available.


  • This is a Read-type Snap.
  • Works in Ultra Tasks when the Batch size is set to 1.

Supported Accounts

Prerequisites

  • Valid Salesforce OAuth 2 account with Read permissions.

Limitations

  • This Snap performs batch processing; a batch of input documents is processed for each HTTP request sent to Salesforce. Values of all the expression-enabled fields must remain constant during the Snap execution or validation. Hence, all expression fields can support Pipeline parameters only when they are expression-enabled. The input data parameters are not supported for expression fields; for example, $serviceVersion.
  • Can execute up to 10,000 query jobs in a 24-hour rolling window.
  • Can store up to 1TB of query results in a 24-hour rolling window.
  • Supports Salesforce Bulk Query service version 47.0 and above.
  • SOQL queries do not support the following commands:
    • COUNT
    • ROLLUPSUM
    • GROUP BY CUBE
    • OFFSET
    • Nested SOQL queries

Snap views

Type Description Examples of upstream and downstream Snaps
Input

This Snap has at most one document input view.

A document containing values required for the bulk query.
Output

This Snap has exactly one document output view.

A document containing the results of the query performed by the Snap.
Learn more about Error handling.

Snap settings

Legend:
  • Expression icon (): Allows using pipeline parameters to set field values dynamically (if enabled). SnapLogic Expressions are not supported. If disabled, you can provide a static value.
  • SnapGPT (): Generates SnapLogic Expressions based on natural language using SnapGPT. Learn more.
  • Suggestion icon (): Populates a list of values dynamically based on your Snap configuration. You can select only one attribute at a time using the icon. Type into the field if it supports a comma-separated list of values.
  • Upload : Uploads files. Learn more.
Learn more about the icons in the Snap settings dialog.
Field/Field set Description
Label

String

Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if more than one of the same Snaps is in the pipeline.

Default value: Salesforce Bulk Query

Example: Salesforce Bulk Query
Service Version*

String/Expression/ Suggestion		 Required. Specify the version number associated with the Salesforce service you want to connect to. Alternatively, click the Suggestion icon to fetch the list of versions and select the desired version.

Default value: 52.0

Example: 41.0
SOQL Query*

String		 Required. Specify the query in SOQL format. You can enter a JavaScript expression that is evaluated each time the Snap is executed. On each document received from the input view, the % symbols will be replaced with the value pointed to by the JSON path $.Id in the input data.
Note:
  • Do not use quotes for field names in the WHERE clause. Using quotes results in an error.
  • Use only single quotes for values in the WHERE clause, as using double quotes results in an error.
  • The above rules do not apply when using SnapLogic expressions; you can use quotes for the field names and values as applicable.

Default value: N/A

Example: "select FirstName, LastName from " + $object + " order by Name LIMIT 5"
Max Records Per Set*

Integer/Expression		 Required. Specify the number of records you want the Snap to fetch in a single set.

Default value: 10000

Example: 5000
Maximum threads

Integer/Expression		 Specify the maximum number of threads to download query results simultaneously in multiple threads. The Snap uses the thread pool to download the query results.
  • The query results are downloaded only if the Maximum Records Per Set value is greater than or equal to 10000.
  • If you expect the query results to be large, we recommend that you increase the values of the Max Records Per Set and Maximum threads for optimal performance.
  • This field does not support upstream values for Maximum threads. However, it supports values from pipeline parameters.

Minimum value: 1

Maximum value: 200

Default value: 10

Example: 9
Polling Interval*

Integer/Expression		 Required. Specify the number of seconds you want the Snap to wait between two queries. At each polling interval, the Snap checks the Bulk API query batch processing status.

Minimum value: 1

Maximum value: 60

Default value: 5

Example: 10
Polling Timeout*

Integer/Expression		 Required. Specify the number of seconds into the query batch execution before the query fails if it receives no response. If the timeout occurs while waiting for the completion of the query batch execution, the Snap throws a SnapExecutionException.

Minimum value: 10

Default value: 3000

Example: 300
Maximum Retry Attempts

Integer/Expression		 Specify the maximum number of retry attempts the Snap must make in case of a network failure.

Minimum value: 1

Default value: 5

Example: 1
Retry Interval (seconds)

Integer/Expression		 Specify the minimum number of seconds the Snap must wait before attempting recovery from a network failure.

Minimum value: 0

Default value: 3

Example: 3
Include Deleted Records

Checkbox		 Select this checkbox to include deleted records in the query result. This feature is supported in Bulk API version 47.0 or later.
Note: This field supports input values from the upstream Snap. However, it does not support pipeline parameters.

Default value: Deselected
Match Data Type

Checkbox		 Select this checkbox to match the data types of the bulk query job results. On selecting this checkbox, the Snap converts string values to the corresponding data types if the original data type is one of Boolean, Integer, Double, Currency, and Percent.
Note: The Salesforce response for null values is independent of the selection of the Match Data Type checkbox.

Default value: Deselected
Display null or empty values as null

Checkbox		 This checkbox controls how empty strings or null data coming from Salesforce are displayed in the output preview.

When you select this checkbox, the empty strings or null data from Salesforce is displayed as null in the output preview.

When you deselect this checkbox, the Snap displays an empty string or null data as blank in the output preview. The null/empty-string values are always displayed as empty ('') string in the output preview.

Note: The output in the Salesforce application remains unaffected. Regardless of the state of the checkbox (selected or deselected), the Salesforce record and its corresponding output continue to appear empty.

Default value: Deselected
Object Type

String/Expression		 Appears on selecting the Match Data Type checkbox.

Specify the object type to match. Alternatively, select an object type from the suggestions list to match the data type.

Note: This Snap does not support Net Zero Cloud Salesforce objects.

Default value: N/A

Example: String
Snap execution

Dropdown list
Choose one of the three modes in which the Snap executes. Available options are:
  • Validate & Execute. Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime.
  • Execute only. Performs full execution of the Snap during pipeline execution without generating preview data.
  • Disabled. Disables the Snap and all Snaps that are downstream from it.

Temporary files

During execution, data processing on Snaplex nodes occurs principally in-memory as streaming and is unencrypted. When processing larger datasets that exceed the available compute memory, the Snap writes unencrypted pipeline data to local storage to optimize the performance. These temporary files are deleted when the pipeline execution completes. You can configure the temporary data's location in the Global properties table of the Snaplex node properties, which can also help avoid pipeline errors because of the unavailability of space. Learn more about Temporary Folder in Configuration Options.

Troubleshooting

Error Cause Resolution
Invalid value: 201 The Maximum thread value is larger than the required maximum, which is 200. Provide 200 or a value lesser than 200 (maximum value) in the Maximum threads field.