Migrating from Legacy-Nested pipelines

The PipeLine Execute Snap can replace some uses of the Nested Pipeline mechanism and the ForEach and Task Execute Snaps. For now, this Snap only supports child Pipelines with unlinked document views (binary views are not supported). If these limitations are not a problem for your use case, read on to find out how to transition to this Snap and the advantages of doing so.

Nested Pipeline

Converting a Nested Pipeline will require the child pipeline to be adapted to have no more than a single unlinked document input view and no more than a single unlinked document output view. If the child pipeline can be made compatible, then you can use this Snap by dropping it on the canvas and selecting the child pipeline for the Pipeline property.

You will also want to enable the Reuse property to preserve the existing execution semantics of Nested pipelines. The advantages of using this Snap over Nested pipelines are:

  • Multiple executions can be started to process documents in parallel.

  • The Pipeline to execute can be determined dynamically by an expression.

  • The original document or binary data is attached to any output documents or binary data if reuse is not enabled.

ForEach

Converting the ForEach Snap to the PipeLine Execute Snap is a simple process. Select the pipeline to run and populate the parameters. The benefits of utilizing the Snap compared to the ForEach Snap are:

  • Documents or binary data fed into the PipeLine Execute Snap can be sent to the child Pipeline execution through an unlinked input view in the child.

    Documents or binary data sent from an unlinked output view in the child Pipeline execution is written out of the Pipeline Execute's output view.

  • The execution label can be changed.

  • The Pipeline to execute can be determined dynamically by an expression.

  • Executing a Pipeline does not require communication with the cloud servers.

Task Execute

Converting a Task Execute Snap to a PipeLine Execute Snap is also straightforward because the properties are similar.

To start, you only need to select the Pipeline you want to use; you no longer have to create a Triggered Task. If you set the Batch Size property in the Task Execute to one, then you will not want to enable the Reuse property. If the Batch Size was greater than one, then you should enable Reuse. The Pipeline parameters should be the same between the Snaps. The advantages of using this Snap to the Task Execute Snap are:

  • A task does not need to be created.

  • You can start multiple executions simultaneously to process documents in parallel.

  • You can process an unlimited number of documents that can be processed with a single execution (that is, no batch size).

  • The execution label can be changed.

  • Determine the pipeline to execute dynamically by an expression.

  • The original document is attached to any output documents if reuse is not enabled.

  • No requirement to communicate with the cloud servers to execute a pipeline.

Converting a pipeline that uses the Router Snap in "auto" mode can be done by moving the duplicated portions of the Pipeline into a new pipeline and then calling that Pipeline using a PipeLine Execute . After refactoring the Pipeline, you can adjust the "Pool Size" of the PipeLine Execute Snap to control how many operations are done in parallel. The advantages of using this Snap over an "Auto" Router are:

  • De-duplication of Snaps in the pipeline.

  • Adjustment of the level of parallelism is trivial simply change the Pool Size value.