Bulk Image Classfication¶
Bulk Image Classification refers to the process of categorizing a large number of images into predefined classes or labels.
To get started, refer below for a quick overview of the Bulk Image Classification project creation process.
For details about the project creation process, refer to Getting Started.
Project Tasks¶
Tensoract Studio provides the following options to browse through the tasks:
Filter Tasks - You can leverage the capabililty to filter tasks based on labels and values to narrow down the results based on specific filter attributes. Following filters are available:
All Fields
Annotations - Count of annotations.
Annotator - Name of annotator worked on the task.
Consensus - Value of consensus.
Final Label - Final label in task.
Dataset - Name of Dataset.
DatasetId - Unique id for dataset .
ItemId - Unique id for each dataset item.
Labels - Annotations from the Completed tasks in the project.
Name - Name of the task/file.
Reviewer - Name of reviewer worked on the task.
State - State of the tasks such as “Unassigned”, “In Progress”,”In review” “Approved” or “Rejected”.
Tags - Tags applied by Admins or Dataset Supervisors to dataset item.
TaskId - Unique id of the task.
Type - Type of task.
Additionally, if there is metadata associated with the tasks, it will also be available in the filters dropdown.
In addition, following options available to refine the task serach and narrow down the results based on specific attributes:
Sort by - Sort the filtered tasks in ascending or descending order.
Image Size - The image scale can be adjusted to increase or decrease the size of the image.
Page Size - The page size scale can be adjusted to increase or decrease the size of the page.
Details (Visible/Hidden) - This control allows you to hide or unhide the available details within the task.
Controls (Visble/Hidden) - This control is associated with Image. This control allows you to hide or unhide RGB controls within the task.
Tasks View¶
Hover Over (5 sec) should show zoomed image of the task in iframe.
Single Click is for multi-selecting tasks.
Double click will open detailed view of the task.
Press and Hold Ctrl key and single click will copy the TaskId of the task.
Project Actions¶
Project task page provides an “Action” button, featuring the following available options:
Export items - Export selected items, filtered items or the entire project.
Delete items - Admin can delete the tasks.
Reset task - Admin can reset the tasks
Add tag - Tags can be added to tasks to select specific tasks in project.
Remove tag - Tags can be removed from the tasks.
Force new label - It allows admin to change the final label of a task along with providing a reason.
The action Force new label is not listed in the Actions button.
For a quick overview of Project Actions,watch the following video:
Labels and Relationships¶
Labels¶
Add list of labels.
After configuring the labels on the page,it seems as follows.
Advanced Settings¶
Settings¶
Annotators per task¶
The setting controls the number of Annotators per task with respect to consensus mode.
Consensus Mode¶
It has a choice to set project with Consensus mode or no consensus mode.
Note
When No consensus mode is selected, then Disable Quality Audit option is available and it follows the one annotator and one reviewer workflow to complete the task.
When Min threshold,2 reviewers is selected, then Consensus Threshold and Min Annotators for consensus options are available.
Consensus Threshold¶
Set Consensus Threshold percentage.
Min Annotators for consensus¶
Select no.of min annotators for consensus.
Number of items per bulk task¶
The setting controls number of items per bulk task and as per that items will be available to Annotators /Reviewers.
Dataset Reference bookmark¶
Create a bookmark within the dataset and set it here for annotators and reviewers to refer in the project.
To view detailed actions for the dataset, click here Dataset-Actions
Exports¶
This sections shows a list of exports, that can also be downloaded by users.
Note
The Exports feature is exclusively available in the Bulk Image Classification project.
Annotator View¶
Once logged into the studio, the annotator is presented with the task queue, which includes a list of tasks available for them to work on.
Bulk Classification Project Dashboard¶
Consensus Label Distribution and Average Consesnus Distribution across all tasks¶
Consensus Label Distribution states the count of labels with respect to Consensus. Average Consesnus Distribution across all tasks gives calculation of the average distribution or proportion of consensus across a set of tasks.
Training Project¶
A Training Project refers to the process of training annotators on the specific annotation guidelines, techniques, and tools required for the project. It aims to familiarize annotators with the project objectives, data types, annotation criteria, and quality standards.
To get started, refer below for a quick overview of the Training project creation process.
For details about the project creation process, refer to Getting Started.
Labels and Relationships¶
Labels¶
Add list of labels.
After configuring the labels on the page,it seems as follows.
Advanced Settings¶
Settings¶
Number of items per bulk task¶
The setting controls number of items per bulk task and as per that items will be available to Annotators /Reviewers.
Dataset Reference bookmark¶
Create a bookmark within the dataset for annotators to refer in the training project.
Training Mode¶
Badge Name¶
A badge name is used to identify and distinguish the specific badge.
Add the Badge Name for training.
Training Type¶
Select Training from the dropdown list.
Upload SOP file¶
Upload SOP file so annotators can refer file while completing the training project.
Upload Training Manual¶
Upload Training Manual so annotators can refer file while completing the training project.
Annotator View¶
While completing Training Project, annotators have the ability to access and review SOP and Manual.
Upon completion of the Training Project, annotator status will be displayed in Dashboard in the project.
Once logged into the studio, the annotator is presented with the task queue, which includes a list of tasks available for them to work on.
Training Project Dashboard¶
It provides the status of team members who have completed the training test.
Label Throughput¶
Qualification Project¶
Qualification Project refers to a task or assignment that helps to assess the skills and capabilities of annotators.It serves as a benchmark or evaluation mechanism to ensure the annotators meet certain quality standards and possess the necessary expertise to accurately annotate the data.
To get started, refer below for a quick overview of the Qualification project creation process.
Labels and Relationships¶
Labels¶
Add list of labels.
After configuring the labels on the page,it seems as follows.
Advanced Settings¶
Settings¶
Number of items per bulk task¶
The setting controls number of items per bulk task and as per that items will be available to Annotators /Reviewers.
Dataset Reference bookmark¶
Create a bookmark within the dataset for annotators to refer in the training project.
Training Mode¶
Badge Name¶
A badge name is used to identify and distinguish the specific badge.
Add the Badge Name for qualification project.
Training Type¶
Select Qualification from the dropdown list.
Badge Expiration¶
Badge expiration refers to the timeframe or validity period associated with a badge.
Select expiration for selected qualification badge.
Success Threshold¶
Set Success Threshold in terms of %.
Upload SOP file¶
Upload SOP file so annotators can refer file while completing the qualification project.
Upload Training Manual¶
Upload Training Manual so annotators can refer file while completing the qualification project.
Admin can see the status of result of Annotator in Dashboard.
In case annotator fails, admin can readd them in Qualification Project.
Annotator View¶
While working on the Qualification Project, annotators have the ability to access and review SOP and Manual.
Upon completion of the Qualification Project and submission of the task, the scores will be displayed for the annotator.
The score obtained is considered valid based on the Badge Expiration setting defined during project creation.
Once logged into the studio, the annotator is presented with the task queue, which includes a list of tasks available for them to work on.
Qualification Project Dashboard¶
The Qualification Project Dashboard provides valuable metrics including Qualification scores distribution,Consensus Label DIstribution,Average Consensus Distribution across all tasks and Label Throughput. To access the Project Dashboard, simply click on the Show Dashboard` button.
Qualification States and Qualification scores distribution¶
The Qualification state pie-chart illustrates the distribution of Qualification states within the project.On the other hand, the Qualification score distribution provides Qualification scores across the project.
Consensus Label Distribution and Average Consesnus Distribution across all tasks¶
Consensus Label Distribution states the count of labels with respect to Consensus. Average Consesnus Distribution across all tasks gives calculation of the average distribution or proportion of consensus across a set of tasks.
Label Throughput¶
It provides the status of team members who have completed the qualification test.
