Building Real-Time Web Applications

Introduction - Twitter Sentiment Analysis

This article shows how to use SnapLogic Ultra Tasks or Triggered Tasks to build real-time web applications, using Tweet sentiment analysis as an example to demonstrate how to measure, analyze, and visualize user sentiment.

SnapLogic Pipelines can be deployed as an Ultra Task or a Triggered Task to provide REST APIs that can power your web/mobile applications using a serverless architecture. Ultra Task provides low-latency by preloading the pipeline in memory. Triggered Task has relatively higher latency since the pipeline loads, executes, or stops for each request (API call), which is ideal for batch processing. For Machine Learning APIs that help build web applications such as Twitter Sentiment Analysis, Ultra Task is preferable to provide a seamless user experience. The following examples use Ultra Task to power web applications:

  • Twitter Sentiment Analysis: Analyzes polarity of a Tweet.
  • Twitter Sentiment Visualizer: Integrates a Twitter API to fetch new Tweets, perform sentiment analysis, and display the result visually.

The following image displays the SnapLogic Pipeline created for the above Twitter web applications:


SnapLogic Pipeline for Twitter Sentiment Analysis

The following image explains the architecture workflow of the above Twitter applications:


Architecture Workflow of the Twitter Sentiment Web Application

  1. Client requests the HTML file. When a user clicks on the web application, the request for the HTML file is sent to the AWS S3 bucket. The HTML file contains the UI layout, components, and JavaScript that performs an action and sends the request to the API.
  2. Client loads the HTML file and renders UI. Client receives the HTML file from AWS S3 and renders the web application containing an input box for the user to add in a Tweet and output box for displaying the response from the sentiment analysis API.
  3. User performs an action and UI sends the request to Ultra Task. The user adds the Tweet in the input box and clicks Submit. Then, the request for sentiment analysis is sent to an API (Ultra Task). This request is passed to the Pipeline's input view that uses a pre-loaded sentiment analysis model. The Pipeline outputs the sentiment of the input Tweet.
  4. Client receives the response from Ultra Task. The sentiment of a Tweet is sent back from Ultra Task and displays in the UI.

SnapLogic Ultra Task provides high reliability and scalability. Ultra Task runs on a Snaplex comprising a cluster of nodes, which can be on-premise or in the cloud; however, we recommend a hybrid approach with 50% of nodes each in the cloud and on-premise. You can specify a number of Pipeline instances to run concurrently. So, unless all nodes go down at the same time, your API will be alive. You can scale the Snaplex horizontally by adding nodes or vertically by using nodes with higher specifications.

Prerequisites

Frontend Development Environment
You can download the HTML file of Twitter Sentiment Analysis from GitHub. You need a text editor to edit the HTML file.
Ultra Task

An Ultra Task in SnapLogic is required to provide an API to your web/mobile application. If you do not already have an Ultra Task, perform the following steps to create one:

  1. Download the Twitter Sentiment Analysis.zip file. This file is an export of a project containing the required Pipelines, tasks, and files.
  2. Import the project. In SnapLogic Manager, select the project space and click Import. For more information, see Importing and Exporting Projects.
  3. Go to your project, click Tasks, and click the task entity to open the Update Task dialog.

    Update Task dialog in SnapLogic Manager

  4. Select the Snaplex from the dropdown list to deploy this Ultra Task on. Click Update.

    Selecting a Snaplex for the Ultra Task

  5. Select the task check box and click Enable in the top right.

    Enabling the Ultra Task

  6. Click the small triangle next to the task name and click Details to open the Task details pane.

    Task menu showing the Details option

  7. Copy the HTTP Endpoint and save it on your local machine to add it later to your HTML file. Ultra Task can take up to 5 minutes to start. You can click Details to view the task status.

    Task details pane showing the HTTP Endpoint

Build Application

Follow the steps below to build your application. You should have the Twitter Sentiment Analysis HTML file and the Ultra Task endpoint before performing these steps.

  1. Test the API (Ultra Task) to ensure it is working as expected.
  2. Modify the HTML file to use your Ultra Task and test it locally.
  3. Deploy your web application.
  1. Test the API (Ultra Task).

    Open the SnapLogic ML API Tester, add your URL and the Body (or use the URL and Body below), and click SUBMIT.

    Note: The URL and token below are examples. Replace them with your own Ultra Task endpoint and token.

    URL: https://<your-env>-fm.snaplogic.io/api/1/rest/feed-master/queue/<your-org>/<project>/Twitter%20Sentiment%20Analysis/Twitter

    Body:

    {
  "token": "snaplogic_ml_showcase",
  "params": {
    "text": "The weather is really nice today."
  }
}

    ML API Tester showing a successful sentiment analysis response

    The sentiment (response from the Ultra Task) displays in a JSON format in the Output section. Thus, the sentiment analysis model gives a positive sentiment of 0.704 (70.4% positive) for "The weather is really nice today" Tweet.

  2. Modify the HTML file and test it locally.

    After verifying that the API is working as expected, you can modify the URL in the HTML file. If you open the HTML file without modification, you are using the default Ultra URL provided by SnapLogic.

    To change the default URL, modify the HTML file at line 43.


    HTML file showing the URL at line 43

    Alternatively, you can remove the Ultra URL and Token settings from the code block above and add your own URL and token in the code at lines 113 and 114.


    HTML file showing the URL and token at lines 113 and 114

    Open the HTML file using a browser and validate that the web application is working as expected.


    Twitter Sentiment Analysis web application running in a browser

  3. Deploy your web application.

    Since the application is an HTML file, you can host the application using your own web server or a static web hosting service. This tutorial uses GitHub Pages. For more information, see GitHub Pages.

    Deploying your Web Application using GitHub Pages:

    You can either follow a tutorial video to understand the process or go through the following steps:

    1. In your GitHub account, navigate to Repositories and click New to open the Create a new repository screen.

      GitHub Repositories page

    2. Provide a name for the new repository and click Create repository. A new repository with the specified name is created.

      Create a new repository form

    3. Go to this repository and click Create new file.

      Repository page showing the Create new file button

    4. Set the file name to index.html and copy your HTML content into the text editor.
    5. Add a commit message and click Commit new file.

      Committing the index.html file to GitHub

    6. Go to the repository settings and browse to GitHub Pages. Under Source, select your default branch (main or master).

      GitHub Pages settings showing the Source branch selection

    7. Click the URL in the GitHub Pages section to access your web application.