Skip to main content

Annotation Review Status

In addition to adding and editing annotations, you can also update an annotation's review status. Review statuses support a variety of workflows:

Quality Assurance Workflow​

One common task is to ensure annotation quality through a review process. In this workflow, an annotator creates annotations, and a separate reviewer later verifies them.

Pre-annotation Workflow​

Another important use case is pre-annotation of datasets using AI models. In this workflow:

  1. Users run an inference job to generate predictions on unlabeled data.
  2. These predictions are then converted into annotations by creating a dataset from those inferences.
  3. The resulting AI-generated annotations automatically receive a review status of Needs Review.
  4. Human reviewers can then verify, reject, or modify these pre-annotations as needed.

This approach significantly speeds up the annotation process by providing a starting point for human annotators while maintaining quality control through the review system.

Data scientists can then choose to train models only using verified annotations. This can be done by creating a View that specifically filters for Verified annotation approval statuses.

Review Status Types​

Chariot currently supports the following annotation approval statuses:

  • Unreviewed: No review status has been set.
  • Verified: The annotation has been reviewed and marked as valid and correct.
  • Rejected: The annotation has been reviewed and marked as incorrect; it either needs adjustment or deletion.
  • Needs Review: The annotation has been flagged for further review. This could be because a human annotator is unsure about the correctness of the annotation or because it was generated by AI and requires human verification.