Incremental
If you enable synchronization when creating the pipeline, for targets that support incremental load, AutoSync does a full load followed by incremental loads (upserts). The Incremental load type requires that each table have a primary key and a timestamp that is updated each time a change is made to a record. For some sources, relational databases for instance, AutoSync requires you to select the column that contains the timestamp. If you do not select a timestamp field for these sources, AutoSync performs full loads when synchronizing.
If your endpoint doesn't already have a column or field to store the last updated timestamp, you will need to add one. If you can't add a timestamp, you can use Full load instead of Incremental.
- Be one of the following types:
datetime
timestamp
- Not be nullable.
- Not contain null values.
For the last updated timestamp field, AutoSync supports values up to nanoseconds. The MySQL destination endpoint is an exception because it doesn't support fractional seconds. The supported data type for the timestamp field differs by endpoint. Refer to the endpoint specific documentation for details.
For most sources, deletions aren't synchronized. However, for Salesforce, you can configure AutoSync to track deletions.
LastModifiedDate
field
because they are not updated after creation. For incremental loads in this case, AutoSync recognizes
new records from their CreatedDate
field and loads them to the destination. The Incremental load type is supported for: Marketo, Microsoft Dynamics 365 for Sales, Oracle, MySQL, Salesforce, ServiceNow, and SQL Server sources when synchronizing to Azure Synapse, Databricks, Google BigQuery, Oracle, SQL Server, and Snowflake targets.