Migrating Artifacts and System Assets

  • Updated on Mar 27, 2025
  • Published on Jan 10, 2025
  • 13 minute(s) read

Artifacts and assets are essential components of Hyperscience. They include flows, releases, Classification, Identification, Transcription, and Finetuning models. Based on your use case, these components work together to enable document processing, model training, and automation or accuracy. Migrating artifacts and assets effectively ensures seamless transitions between environments, maintaining model performance and supporting continuous improvements. This article provides an end-to-end overview of the migration process.

Importance of migration

Artifacts and assets migration is a foundational step to ensure workflows, trained models, and system configurations function optimally across different environments. This process enables the seamless scaling and upgrading of systems, ensuring consistency, efficiency, and high performance in production environments.

Artifact and asset migration typically occurs when transitioning components from a lower environment (e.g., development or testing), to a higher one, such as production. This migration ensures consistency and functionality across different stages of deployment:

  • Environment upgrades - Transitioning from lower training to production environments during system upgrades.

  • Performance optimization - Ensures the latest and most accurate models and workflows are used in production.

  • Data consistency - Maintains consistency of training data, flows, and configurations across environments.

  • Operational efficiency - Reduces redundancy by eliminating the need for manual retraining and reconfiguration in new environments.

  • Testing and validation - Allows testing in lower environments before deploying to production, ensuring robust and error-free processes.

Common scenarios for migration

Artifacts migration is typically needed in the following scenarios:

  • Version upgrades - Moving from one version of Hyperscience to another, especially major upgrades like v35 to v40.

  • Environment changes - Transitioning from a testing environment to a live environment.

  • Workflow scaling - Expanding workflows to handle new document types or processing volumes.

  • Performance Improvements - Implementing updates to improve automation rates and accuracy.

Key artifacts and assets to migrate

Artifacts

  1. Flows:

    • Top-level workflows that determine how documents are processed. Learn more in our Flows Overview article.

  2. Releases

    • Collections of layouts that match the documents you want to process, enabling document classification and extraction. To learn more, see our What is a Release? article.

  3. Classification models

  4. Identification models

    • Models that help the system locate fields and tables in Semi-structured documents. To learn more about identification models, see our Training a Semi-structured Model article.

  5. Transcription models

    • Models that convert handwritten or printed text into machine-readable data for accurate document processing. See Managing Transcription Models for more information.

Assets

  1. Permission groups

    • Groups that control user access by assigning specific roles and permissions to manage what actions users can perform within the system. To learn more, see Permission Groups

  2. System settings

    • The core configurations of the platform.

Migration process

Preparation

Migrating artifacts

Exporting and Importing Flows

Exporting flows

Follow the steps below to export a flow:

  1. Go to the Flows page and find the one you want to export.

  2. Click on the menu () for the flow and click Export All Flows.

When exporting a flow from this menu, all connected subflows are included in the export.

You can also export your flow from its details page:

  1. Go to the Flows page and find the flow you want to export.

  2. Open the flow’s details page and scroll down to the Flows table.

  3. Click Export on the top-level flow.

The exported file is named Flow-[Flow_name]-[Flow_UUID]-[YYYYMMDD_HHMM].zip, where:

  • [Flow_name] is the name of the flow.

  • [Flow_UUID] is its unique identifier.

  • [YYYYMMDD_HHMM] represents the export date and time in GMT.

The ZIP file includes:

  • A flows folder, containing the flow’s JSON file.

  • A manifest.json file with metadata about the export.

  • If the flow contains Code Blocks, their Python files are included.

Note that client secrets and Python package dependencies are not included in the export. For more information, see Managing Flows.

Importing flows

To import your flow:

  1. Go to the Flows page.

  2. Click Import Flow.

  3. Upload the ZIP file you exported.

Exporting and Importing Releases

Exporting Releases

Follow the steps below to export your releases:

  1. Go to Library > Releases, and locate the release you want to export.

  • Use the search box to find the release by name or click Filters to filter the releases by status.

  1. Click the name of the release to open its details.

  2. Click Actions, then select Download Release.

    • The system may take a few minutes to prepare the file. You will receive a notification when it’s ready.

  3. When notified, go to Notifications, find the entry for your file, and click Download.

The downloaded ZIP file contains the following assets:

  • A JSON file containing metadata about the release, such as creation and last updated timestamps.

  • manifest.json, which includes a summary of the release’s contents.

  • field_data_types.json, containing a list of data types used in the release.

  • Individual JSON files for each layout version in the release.

  • Individual JSON files for each layout variation with selected field customizations applied.

Importing Releases

To import your release:

  1. Go to Library > Releases and click Add Release.

  2. Select Upload Existing.

  3. Choose one of the following options:

    • Drag and drop the ZIP file into the dialog box.

    • Click Choose File to locate and upload the file from your machine.

  4. Under How To Handle Duplicate Custom Data Types? select an option:

    • Skip if duplicate custom data type exists - Retains the existing custom data type’s list of values without overwriting.

    • Update with data from the JSON - Overwrites the existing custom data type’s list of values with those from the imported file.

  5. Click Upload.

    • The system may take a few minutes to process the release. You will receive a notification when the process is complete.

    When uploading a release, all included layouts are added to the Layout Library if they do not already exist. If the release was assigned to a flow in the testing environment, it will be automatically assigned to the same flow in the production environment. Field customizations applied to the release are incorporated into the layout’s variations, but they will not appear in the Fields and Customizations tab on the layout’s details page. To learn more, see Adding a New Release.

  6. After the import is finished,  click View Release in the Notification Center to view the release’s details.

Exporting and Importing Classification Models

Exporting Classification Models

You can export Classification models through Training Data Management in v40.1 and later. Learn more in TDM for Classification.

Follow the steps below to export a Classification model:

  1. Go to Library > Models.

  2. Click on Classification Models in the drop-down list at the top of the page.

  3. Click on the name of the model you want to export.

  4. Click the menu () in the upper-right corner of the page, and click Download Classification Model.

    • The system may take a few minutes to prepare the file. You will receive a notification when it’s ready.

  5. When notified, go to Notifications, find the entry for your file, and click Download.

or

  1. On the model’s details page, scroll down to the Model Activity table.

  2. Click the Download model button next to the last-trained version of your Classification model.

The ZIP file contains the following items:

  • Metadata for the model.

  • The model itself.

Note that the model’s training data is not included in the ZIP file.

Importing Classification Models

Classification models can be uploaded only in v40.1 and later and only for live releases. Learn more in TDM for Classification.

  1. Go to Library > Models.

  2. Click Classification Models in the drop-down list at the top of the page.

  3. Click Import Model.

Exporting and Importing Identification Models

Exporting Identification Models

Follow the steps below to export your Identification models:

  1. Go to Library > Models.

  2. Find your Identification model and click its name.

  3. On the model’s details page, scroll down to the Model History table.

  4. Click the download button () in the Actions column to export the model.

The ZIP file contains:

  • Model metadata.

  • The model itself.

Note that the training data is not included in the ZIP file.

Exporting Training Data

To export the training data for your Identification model:

  1. Go to Library > Models.

  2. Open the model you want to export the training data for.

  3. Click Actions, and then click Download training documents… in the drop-down list.

    • The system may take a few minutes to prepare the file. You will receive a notification when it’s ready.

  4. When notified, go to Notifications, find the entry for your file, and click Download.

    You must annotate your documents before downloading the training data.

    • The export consists of a training data ZIP file with the following elements:

      • A JSON file with training data.

      • Document images.

      • Layout version.

A training data ZIP file contains up to 500 pages. If a model has more than 500 pages of training data, the data is split into multiple ZIP files. For example, if a model has 900 pages of training data, two training data ZIP files will be available for download.

Importing Identification Models

Follow the steps below to import your Identification model:

  1. Go to Library > Models.

  2. Click the name of your model to access the model management page.

  3. From the Actions drop-down click Upload Model.

  4. Once the model is uploaded, click Submit.

    You can upload identification models only if their layouts are live. Make sure to match each model to its layout before uploading.

Importing training data

Follow the steps below to import your training data:

  1. After you’ve imported your model, open its details page and click Actions.

  2. Click Upload training documents… in the drop-down list.

  1. Click Upload Existing in the dialog box, and follow the instructions to complete the upload process.

    Clicking Upload Existing allows you to upload training data for layouts that have been created in another environment.  

  2. Once you’re done, click Submit.

    Only one training data bundle can be uploaded at a time.

Exporting Transcription / Finetuning Models

You can export all finetuning models in a Transcription model, or you can download individual finetuning models.

Export all finetuning models in a Transcription model

To export all finetuning models in a Transcription model:

  1. Go to Library > Models, and click on Transcription Models in the drop-down list at the top of the page.

  2. In the Transcription Models table, click on the name of the transcription model whose finetuning models you want to download.

  3. Click Download all models.

The models are downloaded as a ZIP file with the name confidence_thresholding_model_export.zip.

Export an individual finetuning model

To export a specific finetuning model:

  1. Go to Library > Models, and then select Transcription Models in the drop-down list at the top of the page.

  2. In the Transcription Models table, click on the name of the transcription model that contains the finetuning model you want to download.

  3. Click on the tab that matches the type of finetuning model you want to download.

  4. In the Available Models table, find the version of the model you want to download, and click Download Model.

The model is downloaded as a ZIP file with the name confidence_thresholding_model_export.zip

Importing Transcription Models

Importing Transcription models from other versions

Transcription models trained on a different version than your current instance will be imported successfully, but will not appear in the model list. The list only displays models that match the current version of your instance.

Note that you can only import models created in v38 and above. For more information on model compatibility, see Compatibility Across Application, Flow, and Model Versions.

To import a model:

  1. Go to Library > Models, and then click Transcription Models in the drop-down list at the top of the page.

  2. In the Transcription Models table, find the transcription model you want to import a finetuning model for and click on its name.

  3. Click Upload model.

You do not need to click the tab for the type of finetuning model you want to import. The system automatically detects the type of model during the import process.

  1. Do one of the following:

    • Drag and drop the model’s ZIP file into the Import dialog box.

    • Click Choose Files, and find and open the model’s ZIP file on your machine.

  2. Click Import.

Working with Transcription Models

When importing or exporting Transcription models, keep the following factors in mind:

  • Manual vs. Automatic Training

    • Manual Training does not automatically deploy the newly trained Transcription model. Manual training allows you to:

      • Train the model in your production environment (where QA data resides).

      • Export the trained model, test it in a lower environment, and, if satisfied, re-import it into production for manual deployment.

    • Automatic Training, on the other hand, deploys the model immediately upon training. To validate the model before deployment, it’s recommended to disable automatic training.

  • QA Data Migration

    • It is not possible to migrate QA training data from production to a lower environment due to internal safeguards. Instead, you can:

      • Use the manual training method to train and validate models in your lower environment before deployment.

If you are unable to train a Transcription model, check your flows. Ensure that the Transcription Automation Training setting under the Semi-Structured Document Transcription settings type is enabled. To learn more about editing flow settings, see Document Processing Flow in V40.

Transcription model training logic is based on settings from the last-edited flow, so it’s important to confirm this setting is enabled in all applicable flows.

Migrating other assets

Exporting and Importing System Settings

To move system settings between multiple instances that are on the same major version, follow the steps outlined in this section.

Exporting system settings

To export system settings:

  1. Go to Administration > System Settings

  2. Click Export at the top of the page.

  3. Deselect the system settings that you do not want to export.

  4. Click Download Settings.

The system settings are exported in a JSON file. You can then use this JSON file to import these system settings into another instance that uses the same major version of the application.

Importing system settings

To import system settings:

  1. Go to Administration > System Settings.

  2. Click Import at the top of the page

  3. Drag and drop your settings file and click the Import button. You can also click Browse to find the settings file on your machine and open it.

  4. Deselect the settings you do not want to import. All settings that you choose not to import will keep their current value.  

  5. Click Apply Changes now.

Exporting and Importing Permission Groups

Exporting Permission Groups

You can use your custom permission groups in other instances. To do so, export the groups, then import their JSON files into any instances where you would like to use the permission groups.

To export permission groups:

  1. Go to Users > Permission Groups, and select the checkboxes for the groups you want to export.

  2. Click Actions, and then click Export.

The system creates a JSON file for each exported permission group, which is then downloaded to your machine. If you’ve exported multiple groups, the system creates a ZIP file containing a JSON file for each permission group.

Importing permission groups

To import a permission group:

  1. Go to Users > Permission Groups.

  2. Click Add Permission Group in the upper-right corner.

  3. In the Add Permission Group dialog box, click the Upload Existing tab.

  4. Do one of the following:

    • Drag and drop the permission group’s JSON file to the dialog box.

    • Click Choose File, and find and open the permission group’s JSON file on your machine.

  5. Click Upload.

Import / Export tool

The Import/Export page (Administration > Import/Export) provides a centralized interface for migrating configurations, models, and key system data between environments. This feature is designed to simplify the transfer process, ensuring consistency and efficiency.

The following components are available for import/export:

  • System Settings: Export or import global system configurations.

  • Releases: Transfer model training versions or deployment packages.

  • Classification Models: Move machine learning models designed for document classification.

  • Field Identification Models: Migrate models used to extract specific fields from documents.

  • Table Identification Models: Transfer table-extraction models for structured data processing.

  • Field Dictionary: Export or import dictionaries for field mappings.

  • Translations: Migrate translations for multilingual support.

The tool allows you to export or import only one bundle at a time, helping to ensure accuracy and avoid potential conflicts during migration. When exporting or importing models or configurations, make sure that all dependencies are included within the selected bundle to maintain functionality.

Using the Import/Export tool

  1. Go to Administration > Import/Export.

  2. Select the component you wish to import or export.

  3. Click Export to download a bundle or Import to upload one.

  4. Follow any on-screen prompts to complete the action.

Best Practices for Artifact Migrations

  1. Maintain the exact order of migration:

Flows > Releases > Classification Models > Identification Models > Training Data

  1. Disable Continuous Model Improvement.

    • Avoid enabling settings like Continuous Filed Locator Model Improvement or Continuous Classification Model Improvements to prevent performance degradation. Learn more in Identification Settings.

Train models manually in a lower environment before migration.

  1. Preserve high-quality training data.

    • Retain a reliable and validated dataset in a lower environment for model training.

Poor-quality annotations or QA data could pollute the training data, potentially degrading model performance. Carefully evaluate whether to migrate training data or maintain clean datasets exclusively in the lower environment for optimal results.

  1. Validate imported artifacts.

    • After importing, check compatibility and associations:

      • Flows are linked to releases.

      • Models align with respective training data and releases.

  2. Backup old instances.

    • Keep historical data accessible in older environments, if necessary (e.g., a v35 database).

    Validate model performance post-migration to avoid a drop in accuracy or automation. Always retain and reuse high-performing models from your testing environment.