Generate vector embeddings

This example pipeline demonstrates how to generate vector embeddings for sample data.


OpenAI Embedder Example Pipeline

Download this pipeline.
  1. Configure the JSON Generator Snap to pass your input data.
    Note: In this example, we use the JSON Generator Snap. However, you can replace the JSON Generator Snap with any Snap of your choice, such as the Chunker, Constant, File Reader, or S3 File Reader Snaps.

    JSON Generator Snap - Edit JSON

  2. Configure the OpenAI Embedder Snap to generate embedding vectors for input data based on the Model name, batch size, and input document.
    On validation, you can view the input data processed into embedded vectors in the output preview.
    OpenAI Embedder Snap Configuration OpenAI Embedder Snap Output

    OpenAI Embedder Snap Configuration
    OpenAI Embedder Snap Output
    Note: After generating the vector embeddings, you can store the embeddings into a vector database.
To successfully reuse pipelines:
  1. Download and import the pipeline into SnapLogic.
  2. Configure Snap accounts as applicable.
  3. Provide pipeline parameters as applicable.