SQL Server CDC

Overview

You can use this Snap to retrieve data from SQL Server Database transaction logs. It performs Insert, Update, and Delete statements on a specific table within a specified date and time range. Refer to this example to learn how to capture the data changes from the Insert, Update, and Delete statements executed on a table.

It includes the following key features:

Transaction retrieval: The Snap retrieves transactions executed on a table. It can handle Insert, Update, and Delete transactions.
Flexible time range: You can specify the start and end dates for the logs to analyze the data.
Polling interval handling: You can specify a polling interval for the Snap. If a timeout is set, the Snap will process transaction logs at intervals specified by the polling interval.

Supported Accounts

SQL Server CDC Account

This is a Read-type Snap.

Limitations

Works in Ultra Tasks, depending on the SQL Server CDC Snap configuration. If multiple Ultra instances are started with the same configuration, duplicate data might be retrieved. To avoid duplication, we highly recommend that you run only one Ultra task for each pipeline and configuration set.

Prerequisites

A valid SQL Server CDC Account.
User with the SYSADMIN role.

Snap views


Type	Description	Examples of upstream and downstream Snaps
Input	This Snap supports at the most one document input view.	Any Snap with document view.
Output	This Snap supports a maximum of one document output view. The output is a document with all the statement data.	Any Snap with document view.
Learn more about Error handling.

Examples

Retrieve DML statements from table

Snap settings

Note: Learn about the common controls in the Snap settings dialog.


Field/Field set	Type	Description
Label `String`	Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if more than one of the same Snaps is in the pipeline. Default value: SQL Server CDC Example: Employee Table CDC
Database schema `String/Expression/ Suggestion`	Required. Specify the user schema where the table is located. Suggestions are available when a list of users is retrieved directly from the database and accessible users are included. Default value: N/A Example: HR
Table name `String/Expression/ Suggestion`	Specify the name of the database table on which the Change Data Capture (CDC) is performed. Default value: N/A Example: Employees
Transaction type `Dropdown list`	Required. Choose the type of statements to retrieve. The available options are: All: Processes all statements type (Insert, Update, Delete) Insert: Retrieves only insert statements. Update: Retrieves only update statement. Delete: Retrieves only delete statement. Default value: ALL Example: Update
Start time from which to log the data changes `Integer/Expression`	Specify the start date and time for the change logs. The format is: yyyy-mm-ddThh:mm:ss.SSSZ Default value: N/A Example: 2025-01-01T00:00:01.000Z
End time until which to log the data changes `Integer/Expression`	Specify the end date and time for changes. The format is: yyyy-mm-ddThh:mm:ss.SSSZ Default value: N/A Example: 2025-01-31T23:59:59.999Z
Polling mode `Checkbox`	Select this checkbox to continuously poll data. The start and end times defined in the settings enable precise filtering of the data changes captured. Default status: Deselected
Polling interval `String/Expression`	Appears when you select the Polling mode checkbox. Specify the interval to continuously retrieve data changes at specific interval. This mode continuously retrieves data changes at defined intervals. This value represents the frequency of data polling. Default value: N/A Example: 15
Polling unit `String/Expression`	Appears only when you select the Polling mode checkbox. Choose the unit of time for the polling interval. The available options are: HOURS MINUTES SECONDS Default value: N/A Example: HOURS
Last processed lsn file(.json)* `String/Expression`	Appears when you select the Polling mode checkbox. Specify the name of the JSON file that stored the last processed LSN when polling. This file is stored on SLDB. Default value: minLogSequenceNumber.json Example: minLLSNTestDbEmployee.json
Process the response as bulk `Checkbox`	Select this checkbox to display the data in bulk within the `content` object in the output. Default status: Deselected
Number of retries `Integer/Expression`	Specify the maximum number of attempts to be made to receive a response. The request is terminated if the attempts do not result in a response. Default value: 3 Example: 5
Retry interval `Integer`	Specify the time interval between two successive retry requests. A retry happens only when the previous attempt resulted in an exception. The interval is in seconds Default value: 5 Example: 3
Snap Execution `Dropdown list`	Choose one of the three modes in which the Snap executes. Available options are: `Validate & Execute`: Performs limited execution of the Snap and generates a data preview during pipeline validation. Subsequently, performs full execution of the Snap (unlimited records) during pipeline runtime. `Execute only`: Performs full execution of the Snap during pipeline execution without generating preview data. `Disabled`: Disables the Snap and all Snaps that are downstream from it. Default value: Validate & Execute Example: Execute only

Troubleshooting


Error	Reason	Resolution
`No data in the output`	There are no transactions in the defined period.	Verify the specified time range or polling interval to ensure that it encompasses periods with transactions.