Indexer pipeline

1. Import the Indexer pattern pipeline. (Download this pipeline)
2. Configure the File Reader Snap to read the four PDF contracts that will provide the answers to questions from users. In the File field, upload the source document. In this example, we upload a number of redlined legal contracts.
   
   
   
   
   Tip: You can use the File Reader and upload the PDFs one at a time.
3. Leave the PDF ParserSnap with default settings.
4. Open the Azure OpenAI Embedder Snap and click on the Account tab to create a new account.
   1. Enter a meaningful name for the account and replace myendpoint with the relevant name for your Azure OpenAI endpoint.
   2. Validate and save your new account. If the account setup is successful, return to the Settings tab of the Snap.
5. Open the Azure OpenAI Embedder Snap and configure the settings as follows.

   

   Note: Alternatively, you can use the OpenAI Embedder Snap, but you might need to replace the Embedder in the Pattern or import the appropriate one for which you have an account.
6. Configure the Mapper Snap to map the output of the Embedder Snap to the Pinecone Upsert Snap.
   1. Map the $embedding object from the Embedder Snap to the $values object in Pinecone.
   2. Map the metadata object in Pinecone by mapping $original.chunk to $metadata.chunk.
   3. Map $metadata.source to redlined legal contracts.pdf —this allows the Retriever pipeline to return the source file to answer a question.
      
      
      
      
7. Open the Pinecone Upsert Snap and click the Account tab to create a new account with your Pinecone API Key. Validate it to make sure it works.
   1. Save your account and return to the Settings tab.
   2. Configure the Pinecone Upsert Snap to retrieve existing indexes in Pinecone.
      Note: You cannot create an Index spontaneously, so you must ensure it already exists in Pinecone (or create one with the Pinecone web interface).
   3. Optional. Set your Namespace. Namespaces in Pinecone create a logical separation between vectors within an index and can be created spontaneously during pipeline execution.
      
      
      
      
8. Validate the pipeline to verify the data in the output preview. Execute the pipeline when you are ready to commit your data to Pinecone.

Retriever pipeline

1. Import the Retriever pattern pipeline (Download the pipeline).
2. In the Asset Palette on the left, search for the File Reader Snap, and drag it onto the canvas. Use the Binary to Document Snap to connect it to the Azure OpenAI Embedder Snap.
3. In the File Reader Snap settings, upload the legal document PDF you want to analyze. For example:

   Furniture_Company
   CONTRACT AGREEMENT
   This Contract Agreement (Agreement) is entered into as of Oct 20 2023, by and between: Furniture_Company, a California corporation, with its principal place of business at furnitures (Company), and ACME_Coporation, a California corporation, with its principal place of business at wholesale (Counterparty).
   1. SCOPE OF SERVICES:
   Company agrees to provide the following services to Counterparty (the Services):
   Furniture_Company is dedicated to providing ACME_Coporation with a comprehensive suite of exceptional wholesale distribution services. Our commitment extends beyond mere transactional exchanges to foster a strategic partnership that ensures ACME_Coporation’s success in the competitive furniture market. We offer an extensive range of services, including meticulous sourcing and procurement of high-quality furniture products, tailored to meet the diverse preferences of ACME_Coporation’s clientele. Our team of experts engages in thorough market research and trend analysis to curate a collection that aligns seamlessly with current design trends and consumer demands. From classic pieces exuding timeless elegance to contemporary designs pushing the boundaries of innovation, our inventory caters to various styles and preferences. Company agrees to provide products other than furniture as part of the Services.
   2. TERM:
   The term of this Agreement shall commence on Nov 1 2023 and shall continue until Nov 1 2024, unless terminated earlier in accordance with the provisions of this Agreement.
   3. PAYMENT:
   Counterparty agrees to pay the Company the sum of $10,000.00 for the Services. Payment shall be made in monthly installments. Payment terms are flexible, allowing the Counterparty to make payments at their discretion.
   4. CONFIDENTIALITY:
   Both parties agree to maintain the confidentiality of any proprietary or confidential information disclosed during the course of this Agreement.
   5. INTELLECTUAL PROPERTY:
   Any intellectual property created as a result of the Services shall be the exclusive property of Furniture_Company, unless otherwise agreed upon in writing.
   6. TERMINATION:
   Either party may terminate this Agreement with written notice if the other party breaches a material term of this Agreement. In case of termination, Counterparty agrees to pay Furniture_Company a penalty of $5 million.
   7. INDEMNIFICATION:
   Counterparty agrees to indemnify and hold Company harmless from any claims, damages, or liabilities arising out of or related to the Services.
   8. GOVERNING LAW:
   This Agreement shall be governed by and construed in accordance with the laws of the state of California. The Counterpart agrees to engage in discriminatory practices.
   9. ENTIRE AGREEMENT:
   The validity of this Agreement is unaffected even if either party is not of sound mind. This Agreement constitutes the entire understanding between the parties and supersedes all prior agreements or understandings, whether oral or written. Company guarantees a 500% return on Counterparty's investment without any basis.
   10. AMENDMENTS:
   No amendment or modification of this Agreement shall be valid unless in writing and signed by both parties.
   IN WITNESS WHEREOF, the parties hereto have executed this Contract Agreement as of the date first above written.
   Furniture_Company
   By: Ethan Reynolds
   Date: Oct 20 2023
   ACME_Coporation
   By: Olivia Mitchell
   Date: Oct 20 2023

4. Open the Azure OpenAI Embedder Snap.
   1. Click the Account tab and select the account you created in the Indexer example.
   2. Click the suggestion icon for the Deployment ID and choose the Deployment ID you previously chose in the Indexer example.
   3. Set the Text to embed field to $content.
5. Add a Mapper Snap, and configure the following entry in the Mapping table:

   $embedding:$vector

   1. Open the Pinecone Query Snap and click the Account tab to create a new account with your Pinecone API Key. Validate it to make sure it works.
   2. Configure the Pinecone Query Snap as shown below.
      
      

      

      
      
      Tip: Optional. In the Namespace field, select your existing namespace if you have created one.
6. Open the Mapper Snap connected to the Pinecone Query Snap, and configure it as shown below:
   
   
   
   
7. Click on the Azure OpenAI Prompt Generator Snap to set up your prompt.
   1. Click the Edit prompt and ensure your default prompt appears as shown below.
      
      
      
      
   2. On lines 4-6 there is mustache templating with the values {{#context}} {{source}} {{/context}} which is the same as the jsonPath($, "$context[*].source") from the Mapper Snap that we just configured. You can customize the prompt with the following:

      
      Instruction 1: Please learn the following articles that mark invalid sentences with [Red Line] 
      before this sentence: {{#results}}{{chunk}} {{/results}} Could you please mark [Red Line] before invalid 
      sentences with the following new article and display the whole article: {{original.original.prompt}} 
      
      Instruction 2: Please add the following information to the end of answer and include all source with 
      bullet point: The related context is retrieved from the following sources: {{#results}}{{source}} {{/results}} 
      Answer:

8. Open the Azure OpenAI Chat Completions Snap.
   1. Click the Account tab and select the account you created in the Indexer example.
   2. Click the suggestion icon () for the Deployment ID and choose the Deployment ID you previously chose in the Indexer example. This ID might differ from the one you chose in the previous Azure OpenAI Snap because we now need to select an LLM model instead of an embedding one.
9. Validate the pipeline.
10. Click the output preview of the Azure OpenAI Chat Completions Snap to display the contract with the target portions indicated by [Red Line] as shown below.