Incremental

For targets that support incremental load, AutoSync does an initial full load followed by incremental loads (upserts). The Incremental load type requires that each table have a primary key and a last modified field that updates each time a record changes. For some sources, relational databases for instance, AutoSync requires you to select the column that contains the last modified value. If you don't select a column for these sources, AutoSync performs full loads when synchronizing.

To support incremental load, a last modified column must:
  • Be one of the following types:
    • datetime
    • timestamp
  • Be non-nullable and have no null values, unless the environment is configured in Admin Manager to support nulls, as described in AutoSync settings.
If no such column exists, you can:
  • Create it in the source.
  • Use Full load instead. Full load drops the target tables and reloads them on synchronization. It does not require a column that tracks updates. AutoSync uses this load type by default if you do not select a last modified column.\
  • Configure the environment to allow null values on the AutoSync General tab in Admin Manager.

For the last updated timestamp field, AutoSync supports values up to nanoseconds. The MySQL destination endpoint is an exception because it doesn't support fractional seconds. The supported data type for the timestamp field differs by endpoint. Refer to the endpoint specific documentation for details.

For most sources, deletions aren't synchronized. However, for Salesforce, you can configure AutoSync to track deletions.

Important: Some Salesforce objects don't have a LastModifiedDate field because they are not updated after creation. For incremental loads in this case, AutoSync recognizes new records from their CreatedDate field and loads them to the destination.

The Incremental load type is supported for: Amazon Redshift, Marketo, Microsoft Dynamics 365 for Sales, Oracle, MySQL, Salesforce, ServiceNow, Snowflake, and SQL Server sources when synchronizing to Azure Synapse, Databricks, Google BigQuery, Oracle, SQL Server, and Snowflake targets.