Oracle CDC
Overview
You can use this Snap to retrieve data from Oracle Database transaction logs (redo logs). Redo logs record every change made to the data, including insert, update, delete, and structural changes made to the database. It performs Insert, Update, and Delete statements on a specific table within a specified date and time range.
Features
- Transaction retrieval: This Snap retrieves transactions executed on a table through the Insert, Update, and Delete statements.
-
Flexible time range: You can specify the start and end dates for the logs to analyze the data
-
Polling interval handling: You can specify a polling interval for the Snap. If a timeout is set, the Snap processes transaction logs at intervals specified in the polling interval.
Read-type Snap
Works in Ultra Tasks
Prerequisites
- A valid account with the required permissions to use the Oracle Logminer.
- Database to be set up for appropriate logging.
Snap views
| View | Description | Examples of upstream and downstream Snaps |
|---|---|---|
| Input | This Snap supports at the most one document input view. | Mapper |
| Output | This Snap supports a maximum of one document output view. | Mapper |
| Error |
Error handling is a generic way to handle errors without losing data or failing the Snap execution. You can handle the errors that the Snap might encounter when running the pipeline by choosing one of the following options from the When errors occur list under the Views tab. The available options are:
Learn more about Error handling in Pipelines. |
|
Snap settings
- Expression icon (
): JavaScript
syntax to access SnapLogic Expressions to set field values dynamically (if enabled). If
disabled, you can provide a static value. Learn more. - SnapGPT (
): Generates SnapLogic Expressions
based on natural language using SnapGPT. Learn
more. - Suggestion icon (
): Populates a
list of values dynamically based on your Account configuration. - Upload
: Uploads files. Learn more.
| Field / field set | Type | Description |
|---|---|---|
| Label | String |
Required. Specify a unique name for the Snap. Modify this to be more appropriate, especially if more than one of the same Snaps is in the pipeline. Default value: Oracle CDC Example: OracleDB123 CDC |
| Table schema | String/Expression/Suggestion |
Required. Specify the schema where the table is located. Note:
Each schema is uniquely associated with a database user, who serves as the schema's owner. The schema's name is the same as the user's name, and it provides a logical framework for organizing and managing database objects within the Oracle database. Suggestions are available where a list of users is retrieved directly from the database and accessible users are included. Default value: N/A Example: HR |
| Table name | String/Expression/Suggestion |
Required. Specify the name of the database table on which the Change Data Capture (CDC) is performed. Default value: N/A Example: Employees |
| Transaction type | Dropdown list |
Choose the type of statements to retrieve. The available options are:
Default value: ALL Example: Update |
| Start date and time | Integer/Expression | Specify the start date and time for the change logs. The format is: yyyy-mm-dd hh:mm:ss Default value: N/A Example: 2024-01-01 00:00:00 |
| End date and time | Integer/Expression |
Specify the end date and time for changes. The format is: yyyy-mm-dd hh:mm:ss Default value: N/A Example: 2024-12-31-23:59:59 |
| Fetch data | Checkbox | Select this checkbox to fetch the data within the specified time range. When
you define the start and end times, the Snap perfroms precise filtering of the
captured data changes. Default value: N/A Default status: Deselected |
| Bulk output | Checkbox |
Appears only when you select the Fetch data checkbox. Select this checkbox to display the data in bulk within the content object in the output. Default status: Deselected |
| Use archived logs | Checkbox |
Appears only when you select the Fetch data checkbox. Select this checkbox to archive redo log files to retrieve old data. Default status: Deselected |
| Polling interval | Integer | Appears only when you deselect the Fetch Data checkbox. Specify the interval to continuously retrieve data changes at specific interval. This value represents the frequency of data polling. This mode continuously retrieves data changes at defined intervals. Default value: N/A Example: 5 |
| Polling unit | String/Expression | Appears only when you deselect the Fetch data checkbox. Choose the unit of time for the polling interval. The available
options are:
Default value: N/A Example: HOURS |
| Fetch maximum documents per interval | String/Expression | Specify the maximum number of records to be retrieved from the logs. Default value: N/A Example: 100 |
| Last processed lsn file(.json) | String/Expression | Required. Specify the name of the JSON file that stored the last processed LSN when polling. Default value: N/A Example: minLogMinerSequenceNumber.json |
| Redo log files | Use this field set to specify the redo log file paths to be used for retrieving data. | |
| Redo log file path | String/Expression/Suggestion | Specify the redo log file path to be used for fetching data. Default value: N/A Example: C:\ORACLE\DB_19\ORCLSI\REDO03.LOG |
| Snap execution | Dropdown list |
Select one of the three modes in which the Snap executes.
Available options are:
Default value: Validate & Execute Example: Execute only |
Examples
Troubleshooting
No data in the output.
There are no transctions in the defined period.
Verify the specified time range or polling interval to ensure that it encompasses periods with transactions.