Low-Latency feed (request and response) pipeline

Configuration: One unconnected input view and one or more unconnected output views.

• Usage: This common design acts as a data access layer for real-time web services, transforming a pipeline into a continuously running job. Documents are fed into the pipeline through a FeedMaster, which queues the documents. The pipeline processes these documents and returns responses via the FeedMaster.
• Example: A simple Ultra Task Pipeline might include a Mapper Snap that maps incoming documents to a target field. You can download a sample pipeline to create an Ultra Task.

Always-On (continuous data flow) pipeline

Configuration: No unconnected input view and no unconnected output view

• Usage: This design is often used in a listener-consumer construct. An Ultra pipeline with no unconnected views can continuously poll and consume documents from an endpoint without requiring a document feed from the FeedMaster.
• Example: A typical JMS-based polling Ultra Task pipeline might include a JMS Consumer Snap and a Snap that writes data to a destination, such as a File Write Snap. You can download a sample pipeline to create an Ultra Task.

Guidelines for Ultra Task pipeline design

Ultra Tasks function similarly to web service requests and follow an HTTP response architecture. Consider the following guidelines when designing a pipeline to be called by an Ultra Task:

• Avoid spaces in pipeline names that are invoked as tasks.

An unconnected input view type must be either binary or document:

• Binary: HTTP headers are in the binary document's header.
• Document: HTTP request headers are in the root of the document, and the request body is in the content field.

In addition to the HTTP headers in the request, the following fields are added to the input document:

• uri: The original URI of the request.
• method: The HTTP request method.

• query: The parsed version of the query string. The value of this field will be an object whose fields correspond to query string parameters and a list of all the values for that parameter.

{
"foo" : ["bar", "baz"],
"one": ["1"]
}

• task_name: The name of the Task.
• path_info: The part of the path after the Task URL.
• server_ip: The IP address of the FeedMaster that received the request.
• server_port: The TCP port of the FeedMaster that received the request.
• client_ip: The IP address of the client that sent the request.
• client_port: The TCP port of the client that sent the request.

The unlinked output view type can be binary or document as well:

• If the view type is a document:
  • The output document is JSON-encoded and sent back to the client as the response body.
  • To customize the response header and HTTP response code:
    • Map the response body to the $content field of the output.
    • Map the custom header fields to the root $ in the output.
    • Map the custom HTTP response code to the $status field of the output.
  • The other fields in the output document are treated as HTTP response headers to send back to the client.
  • If a content field is not present, the entire document is JSON-encoded and used as the response body to be sent back to the client.
• If the view type is binary data, the binary file's header is sent back to the client as HTTP response headers. The body of the binary file is directly streamed back to the client.
• SnapLogic expects exactly one output document for every input.
• If the output is not sent, then the original HTTP request sent to the FeedMaster hangs, eventually reaching a timeout.
• If more than one output document is generated, then SnapLogic only sends the first one back as the response to the original HTTP request. This behavior is different from a Triggered Task, where all of the documents sent to the unlinked output are sent back in the response.
• Error views are implicitly added to all Snaps when the Pipeline is executed in Ultra mode, regardless of how the Snaps are configured.