How to use AutoSync
SnapLogic AutoSync data pipelines synchronize data between source and destination endpoints. Before creating a data pipeline:
- Gather the credentials and connection information for each endpoint. This information might be stored in the SnapLogic platform already or you can enter it yourself. Learn more.
- Decide which objects, tables, and files to load from source endpoints. Load all of the data required for your use case. Learn more.
- Identify the destination schema. We recommend that you create a new schema in the destination for each AutoSync data pipeline to avoid naming conflicts.
- To connect to on-premises endpoints, understand the requirements. Important: AutoSync can't run in environments with Enhanced Account Encryption. If all of your environments are currently configured for Enhanced Account Encryption, a new environment must be provisioned for AutoSync. In that environment, install a Groundplex that is not configured to use Enhanced Account Encryption.
Open the AutoSync dashboard
The AutoSync dashboard opens:
Use the wizard to create your first AutoSync data pipeline.
Create a data pipeline
To create an AutoSync data pipeline from the dashboard, click Create a new data pipeline:
In the wizard, the first screens ask questions about the purpose of the data pipeline, where to load the data, and which sources to use. You can select up to five sources for one data pipeline. AutoSync loads the data from all sources into the destination schema.
After you select endpoints, if a Groundplex is available, the wizard lets you choose whether to run this data pipeline on the AutoSync Snaplex or a Groundplex. AutoSync can connect to cloud endpoints with either the AutoSync Snaplex or a Groundplex. Some configuration is required to connect to on-premises endpoints.
As you work through the wizard, a description builds dynamically in the right side panel. The description documents your choices and makes it easy to share the pipeline's purpose with others. For example:
When you add endpoint credentials, a link on the icon indicates the connection status. For example, the green link in the screenshot shows that AutoSync successfully connected to Snowflake:
Table names must be unique in the destination schema. If you select tables or objects with the same name from multiple sources, AutoSync suggests unique names for the destination tables by appending the source type to the table name. You can accept the suggested names or enter your own.
Next, transformation screens give you the option to transform data before loading it. Learn more about the available transformations.
At the end of the wizard, you can schedule synchronization, save the data pipeline without running it, or run it immediately. After saving, AutoSync adds the data pipeline summary card to the dashboard. Environment admins (Org admins) can view all data pipelines in the Environment from the dashboard. Non-admin users can view data pipelines:
- Created by them
- Shared with them
- When ownership is transferred to them
View the data pipeline summary
The AutoSync dashboard displays a summary card for each data pipeline:
Click the card to view the details panel. From the details panel:
- View the full description.
- Edit, delete, or run the data pipeline.
- View the number of tables loaded and for the last run, the number of records processed, whether new, updated, or failed.
The following shows an example details panel:
Use the dashboard list view
In the AutoSync dashboard, click the list view button:
The list view opens:
From the list view, you can:
- Filter data pipelines to include all, only yours, favorites, or shared.
- Search.
- Switch back to card view.
- Access settings.
- Create a new data pipeline.
- View the details panel by clicking a data pipeline. From the details panel, you can edit or run the data pipeline and view statistics.
- Edit a data pipeline by selecting it and clicking the edit button on the right.
- Delete one or more data pipelines by clicking the box to the left of the name.
- Enable or disable synchronization.
- Click the star icon to add as a favorite.
- Run now, share, or delete from the three dot menu that displays when you hover on the right side of a row.
- Sort by creation date in ascending or descending order.