Automation of trained models is one of the metrics that can help you determine the success of your use case.
In this article, you will learn more about automation and its dependencies.
What is Automation?
Automation (also known as Skip Rate) refers to the extent to which a machine can process data independently, without requiring human supervision. It represents the proportion of extracted data with confidence scores exceeding a specified threshold. This threshold is determined by the level of accuracy you want your extracted data to have.
Automation represents the percentage of the work done by the machine for
Classification models
Identification models and
Transcription models.
Types of automation
Projected automation
Projected automation is the predicted automation based on the desired target accuracy. It's calculated from a portion of the training data used for the model training process. This metric helps estimate how much automation can be achieved while aiming for specific performance goals. Learn more about model training in Training a Semi-structured Model.
Actual Automation
The actual automation represents the real automation achieved through processing the data in your specific use case. Learn more about your model’s performance in Evaluating Model Training Results.
Accuracy and automation
There is a dependency between accuracy and automation. When your model has increased automation, more fields, including some with lower confidence, are transcribed without human oversight. On the other hand, when higher accuracy is needed, automation decreases because even high-confidence fields are reviewed by humans to ensure precision. See our Accuracy article for more information.
Automation reports
The Automation chart reports the percentage of automation for a given date range.
Field Transcription Automation and Table Transcription Automation percentages at the bottom of the chart report the amount of transcription automation for the selected date range. Machine Field Transcriptions report the number of machine-transcribed fields.
To see additional performance metrics, hover your cursor over the data point of interest and a window will display the percentage of automation (Transcription, Identification, or Table Identification), the number of machine-extracted fields, and the total number of fields.
Note the following about the Automation report:
For Field Identification automation – the chart reports on an occurrence level rather than on a field level.
For Table Identification automation – the percentage is composed of the number of columns correctly predicted out of the total columns that the machine predicted.
Filtering options
The following filtering options are available in the Automation chart:
Date range: by default, these figures are generated with the last week's data, but can be recalculated for a custom date range.
Task type: Transcription, Identification, or Classification data can be reported.
All Layout Variations: Choose a specific layout you want to see the automation for.
All Flows: Choose a flow you want to see the automation for.