Write up to Three Input Documents into SLDB and Reject All Additional Documents

In this example, you have a list of documents, and you want to filter the first six input documents based on the number they contain in a specific field. You want all odd numbers routed into one branch, and all the even numbers, into another. To do so, you create a parent and child Pipeline.

  • The parent Pipeline accepts multiple documents and triggers the child Pipeline for each document.
  • The child Pipeline routes incoming documents into two branches based on their contents. It then accepts only the first three documents in each branch and generates an error for each additional document.
The Child Pipeline
You first design the child Pipeline.

Download this Pipeline

  1. Add a Router Snap to the Pipeline and configure it to route all documents containing even numbers in the $num field into a branch labeled Even, and all documents containing odd numbers into a branch labeled Odd.

  2. Add an Exit Snap to each output view of the Router Snap and configure them both to pass three documents and throw an error when they receive more than three documents. Also, add an output view to each Exit Snap, so you can view the documents that were passed.

  3. Add a JSON Formatter Snap to the output view of each Exit Snap to convert the Exit Snap's output documents into a binary stream of data.
  4. Add a File Writer Snap to the output view of each JSON Formatter Snap and configure each to write out the data received as JSON files in SLDB.




The Parent Pipeline
Design the parent Pipeline to accept multiple documents and trigger one child Pipeline execution for all the input documents, together.

Download this Pipeline

  1. Add a JSON Generator Snap to the Pipeline and configure it send seven documents, in three of which the $num field contains even numbers. In the four remaining documents, $num contains odd numbers.

  2. Add a PipeLine Execute Snap to the Pipeline and configure it to execute the child Pipeline once for the entire collection of documents received from the JSON Generator Snap. To do so, select the Reuse executions to process documents check box. If you do not select this check box, then the child Pipeline will be executed once for each document coming in, and the Exit Snap will always receive only one document and never throw an error. For details on configuring the PipeLine Execute Snap, see PipeLine Execute .

  3. Save and execute the parent Pipeline.
    1. The PipeLine Execute Snap sends all seven documents together to the child Pipeline for processing.
    2. The Router Snap routes the incoming documents into two branches, Even and Odd.
    3. The Exit Snap attached to the Even branch offers all three documents as output and does not throw any error.
    4. The Exit Snap attached to the Odd branch offers all three documents as output. It also sends an error back to the PipeLine Execute Snap, which triggered the child Pipeline.

    5. The two File Writer Snaps create two documents, Evens.JSON and Odds.JSON, in SLDB, each containing three documents.




Downloads

Note: To successfully reuse pipelines:
  1. Download and import the Pipeline into SnapLogic.
  2. Configure Snap accounts as applicable.
  3. Provide Pipeline parameters as applicable.