Scoring Field Identification Accuracy

Overview

In order for the system to score the accuracy of the machine or a data keyer, a field must first achieve Quality Assurance (QA) consensus; consensus means that two identifications for the field must be matching. In other words, when the field ID of the machine (with high confidence) or that of a data keyer matches what is determined in QA consensus, then the data keyer or the machine is marked as accurate.   The chance of two locations matching exactly is nearly impossible. As such, we ask users to confirm the location of the previous entry in order to establish consensus rather than drawing a new box every time. 

Scoring Field Identification

With release 25.0.0, the system scores every human entry for Field ID. However, consistent with previous releases, there are specific instances where this is not the case:

• When a human marks a field as illegible (e.g. abandons it), the field is not scored.

• When the machine is confident in the location, but during regular SV Field ID, a human deems the location is incorrect, the machine will be scored as wrong.

The major difference between Transcription QA and Field ID QA is that Field ID QA is sampled and done at the page level whereas Transcription QA happens at the individual field level.

Here are a few example scenarios:

Example 1:

 Example 2:

 Example 3:

Example 4:

Abandoned fields

There is one situation in which the affected entries will be abandoned and therefore not counted towards accuracy reporting:

1. A QA entry is abandoned if no consensus is reached on location after three QA entries.

   • Although an abandoned QA entry is not counted towards accuracy metrics, the QA entry does count towards a keyer’s productivity