Field ID and Table ID Model Validation Tasks

Overview

During model training, a model may discover fields where it is very sure of the field's location, but based on data – either from Table ID Supervision/QA or Field ID Supervision/QA – that the location has been marked incorrectly. This will lead to lower automation because the system will learn that it has made errors with high confidence. To fix this problem, you must complete Model Validation Tasks.

Model Validation Tasks (MVTs) ask you to verify whether a field or a table cell location is accurate or not. 

  • If a field has multiple occurrences, all occurrences are pulled into the MVT. 

  • You can add and remove field occurrences during MVTs.

  • If a field has multiple bounding boxes, all bounding boxes are pulled into the MVT.

  • You can add and remove multiple bounding boxes during MVTs.

  • MVTs support both regular and nested tables.

  • Table cells with dashed yellow borders are cells that keyers and the machine did not agree on.

If you re-train a model using the results from MVTs, you will improve automation.

  • MVTs are used for both Field ID and Table ID models.

  • After re-training, the new Candidate Model may create additional MVTs.

  • MVTs are tied to trained candidate models. When a new model is trained or imported, the old MVTs tied to the previous model will be deleted.

  • Performing subsequent tasks and retraining without additional documents may not lead to better automation.

The general process:

  1. Train a model

  2. Perform MVTs

  3. Re-train a model with corrected data

MVTs are identical to Field ID QA and Table ID QA tasks. You can access and perform MVTs from the Model Details page:

  1. Go to Library > Models.

  2. Click a layout’s name from the Locator Models table.

  3. Click the Perform Tasks button at the top of the Model Details page.

After completing MVTs, the model will automatically retrain with the updated training data at the next regularly scheduled training interval. You can also manually start the training of models. 

Best practices

  • If the bounding box is less than one character off, do not adjust it. This will avoid overcorrection.

  • You do not need to complete all MVTs in order to retrain a model. Within the Actions dropdown, click "Keep Live Model" and then "Run Training".

  • While completion of tasks does not block model deployment, it is recommended to complete at least one round of Model Validation tasks prior to deploying a model.

  • After completing subsequent rounds of MVTs with the same QA data, marginal performance gains will be reduced. For example, if all the MVTs require that you mark the field as incorrect and draw a new box, then you have reached a performance plateau. We typically see this after 1-2 rounds.

Additional notes

  • These tasks cannot be moved from one instance to another.