Model Integration

Tensoract Studio supports integration with custom models that can be leveraged for pre-labeling,auto-labeling of tasks in a project, document extraction in datasets and utilization as prompts. Following are the steps to add a custom model to the workbench.

  1. Navigate to Models on the left navigation bar.

    Alternative text

  2. Click Add Model to bring up the model setup screen.

  3. Currently supported model types are as below

    • NER Labeling - used to pre/auto label tasks in a NER project.

    • OCR Labeling - used to pre/auto label tasks in OCR project.

    • Bulk Image Classification-Labeling - used to pre/auto label tasks in Bulk Image Classification project.

    • OCR - used for extraction for documents in Scanned(OCR) datasets.

    • PII Redaction - hide the PII/replace with characters ** in documents in dataset.

    • Classification (prompt)- classify the documents in dataset.

    • Summarization (prompt) -summarize the documents in dataset.

    • Q&A (prompt) - question-based input given to a model for generating relevant answers or information.

    • Text Embeddings - converts words or phrases into numerical vectors in a way that captures semantic meaning.

    • Image Embeddings - transforms images into numerical vectors that encode visual characteristics.

    • Video Embeddings - generates compact numerical representations of videos.

    • Audio Embeddings - creates condensed numerical representations of audio signals.

1. Labeling models and OCR model

The subsequent steps outline the process of configuring the labeling and OCR models.

  1. Select the model type as NER Labeling/OCR Labeling/ Bulk Image Classification-Labeling/OCR Model.

  2. Provide a name for the custom model.

  3. Specify the Rest API Endpoint URL for the custom model. Please note that Tensoract Studio should have network connectivity with this endpoint URL for the custom model integration to work properly.

  4. Specify any optional model parameters. For example, you can include model authentication parameters as key-value pairs in the Header.

  5. You can test the model integration with the default payload, which can be customized. Click Test and validate the model output to ensure the integration works as expected.

    For Model request payloads and response , please refer to this section Model Integration: Request Payloads and Responses.

  1. Configure one or more Labels and Relationship for the model.

  2. Click Save Model and the model is now ready for use in projects for pre/auto labeling of tasks.

Refer to the following video for an overview of custom model integration steps:

_images/Create-model-1.gif

2. Classification, Summarization and Q&A Models

Three distinct platforms, labeled OpenAi, Amazon Bedrock and Meta offer various models for configuring the setup of Classification, Summarization and Q&A models.

1. Model Setup with OpenAi platform

  1. Model Info

    • Model Type - Select model type as Classification, Summarization and Q&A .

    • Model Name - Provide a name for the model.

  2. Model Parameters

    • LLM Platform - Select OpenAi as LLM platform.

    • LLM Models - This menu will show the models linked to the platforms.Select one of the models. List of models-

      1. GPT-4

      2. GPT-3.5-Turbo

      3. Davinci

      4. Curie

    • openai_api_key - Set OpenAi key.

  3. Default Temperature (0-1)-Enter temperature value. The Temperature setting in an LLM controls the randomness of its output.A higher temperature makes the output more creative but less focused, while a lower temperature makes the output more predictable and controlled.

  4. Max. number of concurrent invocations - establishes an upper bound on the number of simultaneous operations allowed.Set max. number of concurrent invocations

  5. Test Model

    • Click Test Model and validate the model output to ensure the integration works as expected.

  6. Click Save Model and the model is now ready for use in datasets.

2. Model Setup with Amazon Bedrock platform

  1. Model Info

    • Model Type - Select model type as Classification, Summarization and Q&A .

    • Model Name - Provide a name for the model.

  2. Model Parameters

    • LLM Platform - Select Amazon Bedrock as LLm platform.

    • LLM Models - This menu will show the models linked to the platforms.Select model.

    List of models-

    1. Amazon Titan TG1 Large

    2. Anthropic Claude V2

    3. Jurrasic-2-Ultra

  3. Default Temperature (0-1)-Enter temperature value. The Temperature setting in an LLM controls the randomness of its output.A higher temperature makes the output more creative but less focused, while a lower temperature makes the output more predictable and controlled.

  4. Max. number of concurrent invocations - establishes an upper bound on the number of simultaneous operations allowed.Set max. number of concurrent invocations

  5. Test Model

    • Click Test Model and validate the model output to ensure the integration works as expected.

  6. Click Save Model and the model is now ready for use in datasets.

3. Model Setup with Meta platform

  1. Model Info

    • Model Type - Select model type as Classification, Summarization and Q&A .

    • Model Name - Provide a name for the model.

  2. Model Parameters

    • LLM Platform - Select Meta as LLm platform.

    • LLM Models - This menu will show the models linked to the platforms.Select model.

  3. Default Temperature (0-1)-Enter temperature value. The Temperature setting in an LLM controls the randomness of its output.A higher temperature makes the output more creative but less focused, while a lower temperature makes the output more predictable and controlled.

  4. Max. number of concurrent invocations - establishes an upper bound on the number of simultaneous operations allowed.Set max. number of concurrent invocations

  5. Test Model

    • Click Test Model and validate the model output to ensure the integration works as expected.

  6. Click Save Model and the model is now ready for use in datasets.

3. PII Redaction

The PII redaction model automatically detects and masks personally identifiable information (PII) from text, ensuring data privacy and security.

  1. Model Info

    • Model Type - Select model type.

    • Provide a name for the model

  2. Model Parameters

    • Max. number of concurrent invocations - establishes an upper bound on the number of simultaneous operations allowed.Set max. number of concurrent invocations

  3. Test Model

    • Click Test Model and validate the model output to ensure the integration works as expected.

4. Embedding Models

  1. Model Info

    • Model Type - Select model type as Text Embedding | Image Embedding | Video Embedding | Audio Embedding .

    • Model Name - Provide a name for the model.

  2. Model Parameters

    • Embedding Model - The drop-down menu will display a list of embedding models, as outlined below. 1. mpnet 2. e5_base_v2 3. e5_large_v2

  3. Max. number of concurrent invocations - establishes an upper bound on the number of simultaneous operations allowed.Set max. number of concurrent invocations

  4. Text Chunk Size

  5. Text Overlap Size (between chunks)