SQL Server Bulk Load as a target
The SQL Server Bulk Load endpoint provides better performance for data pipelines that synchronize
large datasets. This endpoint requires the data pipeline to run on a Groundplex configured to use the
Microsoft SQL Server Bulk Copy Utility, bcp
. Before using this endpoint in an
AutoSync data pipeline, you must install bcp
on each Groundplex node.
For testing and troubleshooting, you can have AutoSync create error tables to store error records.
Install and configure bcp
To install SQL Server bcp
:
- On each Groundplex node host, follow the instructions for the appropriate operating system:
- On Windows systems:
- If not already installed, download and install the ODBC Driver for SQL Server.
- Download the Microsoft Command Line Utilities 15 for SQL Server for X64 processor or X86 processor.
- Install by running the downloaded .msi file.
- On Linux systems: Follow the steps to install the SQL Server utilities.
- On MacOS systems: Follow the steps to install the SQL Server utilities.
- On Windows systems:
- To verify the installation, at a command prompt enter:
A successful installation returns the version number.bcp -v
- Note the location of the bcp installation folder.
- Add the PATH to the
bcp
installation to the host environment variables, using the appropriate steps for the operating system:- On Windows systems, open the Environment variables dialog from the Control Panel or from a command window, use the set command.
- On Linux or MacOS systems, add the path to the startup configuration file, such as .bashrc or .zshrc.
- Restart the Groundplex nodes.
Supported Snap account types
Designer and Classic Manager provide multiple Snap account types for most endpoints and not all types are compatible with AutoSync. When you create or edit a data pipeline, the existing credentials list includes only compatible accounts.
Many Snap accounts support references to JAR files. If you create an account that references a JAR file for use in AutoSync, you must use an absolute path to the JAR file.
For SQL Server, AutoSync supports:
- SQL Server Account
Connection configuration
To configure SQL Server Bulk Load in AutoSync:
- Enter the following properties:
- Credential label: A unique, meaningful name such as
SQL-Server-Sales
. If a configuration with the same name exists, AutoSync displays anAsset conflict error message
. - Hostname: The name of the machine hosting the database.
- Port number: The port number to connect to the
database. Default value:
1433
- Database name: The name of the
database. For example,
MyDB
. - Username: A username for an account with the correct permissions for AutoSync to synchronize data.
- Password: The password for the account. Multiple retries with an invalid password can cause your account to be locked.
- Share: (Optional) Select a user group to share this configuration with. Org admins create user groups to share credentials. If you are a member of a user group, you can select it from the dropdown. You can also select the global shared folder, which shares the credentials with everyone in your Org.
- Credential label: A unique, meaningful name such as
- Validate and save: After saving, AutoSync adds the configuration to the list of saved credentials.
- Select schema: AutoSync populates this list from the account. Choose the schema to use as the destination.
- Select Log errors in a separate table to have AutoSync save errors. This can impact performance. Refer to Error tables for more information.