Question & Answer¶
Question and Answer(Q&A) is the task of building well-formed questions and their corresponding answers.
Watch below video for a quick glimpse into Q&A project creation. Each step and option is detailed further below on this page.
Create Project¶
On the Projects menu in left-side navigation pane, Click Create New Project
Enter a unique Project Name and a relevant Project Description
On the project type drop-down, select Q&A and Click Next
Label & Relationships¶
COnfigure the Question, Answer and Summery Formats
Distribution - The distribution of labels is uniform or non-uniform based on that value.
After configuring the question and answer on the Labels page, it seems as follows.
Advanced Settings¶
The Advanced Settings section has some useful features which would drive the task workflow behaviour. Each setting is explained in detail below.
Disable Quality Audit¶
If this option is unchecked, then tasks submitted by the annotator will be in Submitted for review state and has to be reviewed by the QC /Reviewer. If this option is checked, then there is no reviewer in the workflow. Upon submission by the annotator, the task will transition directly to Submitted state.
QC may request adjustment¶
- This option will be visible only if the Disable Quality Audit option is unchecked. When the QC may request adjustment is unchecked, then below two options are available for the reviewer to choose in the task.
Annotations are ok - when reviewer selects this option, the task will transition to Approved state.
Reject annotations - when reviewer selects this option, the task will transition to Rejected state.
Break text Files in pages¶
It breaks the Text files into pages depending on the Number of Words per Page
Request text highlight for Question Annotations¶
Request text highlight for Answer Annotations¶
Request text highlight for Sumarization Annotations¶
Note
If you want to skip the Question/Answer/Summary Annotation then Disable Pen icon
List all tasks to Annotator/Review Team¶
If Full view is set, all tasks from the project will feature in the annotators jobs page as below. The user might view all tasks but some may be locked depending on ownership and task state.
Maximum concurrent tasks per annotator¶
This setting controls the maximum number of concurrent tasks that will be allotted per annotator.
Note
When List all tasks to Annotator/Review Team setting is enabled, then Maximum concurrent tasks per annotator option will not be available.
Project Metadata¶
Under Project Metadata, Labels and Values can be set and these metadata values will be preserved in the task, export mainifest file and project descriptor json file. This can be leveraged to track and filter useful metadata related to the project.
Webhook¶
Webhooks are automated messages that can be sent to a configured server URI in response to a specific event. Currently, webhooks are supported for Task update and Project update.
Browse tasks¶
This section will display a summary of tasks in the selected project.
Annotator View¶
Shortcut Keys
The following shortcut keys serve as helpful references for annotators and reviewers during the task annotation process.
Keys |
Use |
|---|---|
Ctrl + F or F3 |
Find Text |
Ctrl + G |
Find Next |
Ctrl + ] |
Move to next entity |
Ctrl + [ |
Move to previous entity |
Ctrl + } |
Move to next entity of same type |
Ctrl + { |
Move to previous entity of same type |
Ctrl + Arrow Right |
Move to next page |
Ctrl + Arrow Left |
Move to previous page |
Ctrl + |
Zoom In |
Ctrl - |
Zoom Out |
Ctrl + 0 |
Zoom Reset |
Alt |
Hide Annotations, Label popup, Relationship |
Alt + Entity Select |
Enable Entity Comment |
Delete |
Delete selected entity/relationship/box |
When an Annotator logs into to the workbench, the task queue is displayed with a list of tasks that the Annotator can work on.
Watch the below video to a feel of the Annotator view and the options available to the Annotator to accurately and efficiently label the tasks.
