When planning your upgrade, you should be aware of how the upgrade will affect your Hyperscience experience, along with any issues you may encounter during the upgrade process.
Before upgrading, we encourage you to review the sections below, followed by our Upgrade Best Practices.
Supported versions
We release new versions of Hyperscience several times a year. We support the three most recent versions of Hyperscience, so we recommend updating regularly to maintain support and take advantage of our latest features.
Compatibility across versions
Some aspects of your current Hyperscience configuration may not be compatible with your target upgrade version. To learn more about compatibility across versions, see the sections below.
Application
Application bundles are compatible only with the previous bundle (i.e., version N-1 bundles). You cannot skip versions because the application needs all flows and blocks from previous versions to exist. Even if you don’t want to use a given application version, you still need to unpack its bundle. Unpacking a version’s bundle allows our post-migration steps to run.
The only time you can skip an application version is when upgrading from v28 to v31 while skipping v30, which is possible.
Trainers
Trainers cannot be upgraded to newer versions. Instead, you need to install a new trainer version when upgrading. For ease of use, you can install the new trainer version on the same machine where the previous trainer version is installed. To learn more about using two trainers on the same machine, see the “Using trainers on the same machine” section in Upgrade Process Overview.
You cannot install a trainer on the machine where the Hyperscience application is installed.
If you have two trainers on the same machine:
Stop the old trainer.
Initiate future training on the new trainer version.
Wait until your future training is completed.
Once the application is successfully upgraded, remove the old trainer.
Submissions
Submissions that are uploaded before an upgrade are compatible with the next two versions. For example, submissions that are uploaded in v32 are compatible with v33 and v34. If you upgrade to v35 and still have submissions that were uploaded in v32, you have to resubmit these submissions or drain them. To learn more about how to drain submissions, see the “Draining submissions” content in our version-specific upgrade articles:
Layouts and releases
Layouts and releases are compatible with all future versions. After upgrading the application, all layouts and releases you had on your previous version continue to function as intended.
After upgrading, layouts and releases preserve their respective statuses:
Layout statuses:
Live
Not Live
Release statuses
Live
Locked
Draft
Models
You need to retrain models on each major version. Models are not compatible with future or past major versions. For example, models trained with a v31 trainer cannot be used in a v32 application. If you export a model trained with a v31 trainer and try to import it to a v32 application, you will receive an error message, and the operation will fail.
Unless otherwise noted, you do not need to retrain models when upgrading to a minor version (e.g. v33.0.0 to v33.1.0).
Training models
A trainer can use data from the application version it is bundled with, along with the previous two application versions. For example, you can run training jobs on a v32 trainer with data from v30, v31, and v32 applications. However, the output of a trainer can only be used with the application version it is bundled with. For example, when you run training jobs on a v32 trainer on a v31 application, you will only be able to use that trainer's output after upgrading the application to v32.
As part of the upgrade process, you need to attach the target upgrade version of the trainer to your current application. To learn more about attaching a new trainer to your application, see the “Step 1: Install the new trainer” section in Upgrade Process Overview.
We recommend allowing the new trainer to run training jobs for at least 2-3 days before upgrading the application. You may benefit from allowing even more time for training, depending on the number of eligible training documents available. Only submissions that have not yet had their PII deleted can be used to train the models. Contact your Hyperscience representative to determine how long to train the models on the new trainer before upgrading the application.
Connectors
Connectors are only available in Hyperscience v28 or earlier. In v30 and later, Input Blocks and Output Blocks are used in place of connectors.
Version-specific considerations
Before upgrading, we recommend reviewing the release notes of your target upgrade version, along with any versions between that version and your current version. You can find these release notes in the Release Notes section of this site.
In addition to the features mentioned in the release notes, some versions of Hyperscience include changes to the user experience or to certain settings. The sections below describe changes that you should be aware of before upgrading.
Upgrading from v33 to v35
In v35, you cannot use both v33 and v34 flows at the same time. Upgrading from v33 to v34 invalidates your Field Locator and Table Locator models. If you do not retrain these models in v34 and just use v34 as an intermediary step before upgrading to v35, you will lose all your automation.
If you want to continue using your v33 flows in v35, you must skip v34 in the upgrade process. Directly upgrade to v35 by deploying v35 on all machines running the application.
If you do not want to use your v33 flows and instead want to use v34 flows in v35, you need to upgrade to v34 before upgrading to v35.
If you want to go through v34 but use your v33 Field Locator and Table Locator models, you need to import your v33 Field_Locator and Table_Locator blocks in /admin. To do so:
Go to /admin/sdm/blockdefinition/ and click Import from ZIP.
Click Choose Files, and select the files from your machine that you want to import.
Select the Replace option. You need to select this option because existing blocks have the same names as the ones you are importing.
Click Upload File.
Upgrading to v31+
Trainer API Users
We added a Trainer API User permission group in v31. By default, the system includes all users with the API Access permission in this group. However, to maximize the security benefits of the Trainer API User permission group, we recommend removing all users from this group other than the user you’ve created for the trainer.
To learn more about permission groups, see the “Permission Groups” article for your version of Hyperscience ( v35 | v36 | v37 | v38 | v39 | v40 ).
Upgrading to v30+
Polling interval for block tasks
The SDM_BLOCKS_TASK_POLL_INTERVAL ".env" file variable allows you to adjust the polling interval for block tasks, which affects the overall throughput of your system. In general, the fewer submissions you expect to process in an environment, and the fewer VMs an environment has, the lower the value should be.
Latency in submission processing
To support flows and other customizable features in v30 and later, we needed to make changes to the application architecture used in earlier versions. As a result of these changes, you may notice increased latency in submission processing after upgrading to v30 or later. However, while each individual submission may take longer to process, the throughput, or total number of submissions that can be processed within a given time period, is greater in v30 and later than in earlier versions.
To control how quickly the system processes submissions, you can include the SDM_BLOCKS_TASK_POLL_INTERVAL variable in your ".env" file. For more information on configuring this variable, see Task-Polling Intervals.
Upgrading from v30 to v31
Flows
When upgrading from v30 to v31, the system will automatically migrate your v30 flows to v31. No changes will be made to your flows as part of this process. Submissions will continue to be processed through the "Document Processing" flow as they were in v30. This flow will be active and deployed in v31 by default.
The system will also migrate your "Document Processing Notifications" flow to v31, but you will not be able to enable or disable connections in Output Blocks in the application. Contact your Hyperscience representative if you need to enable or disable output connections in this flow after upgrading to v31.
A new, uncustomized "Document Processing (V31)" flow will be added and will be disabled. This flow is a modified version of the default "Document Processing" v30 flow that contains blocks introduced in v31. You can work with your Hyperscience representative to determine if this updated flow would be beneficial for your business. If you decide to use the new flow, you will need to add Input Blocks and Output Blocks to it, as they are not included by default.
Upgrading from v30.0.7
To prevent data-migration issues when upgrading from v30.0.7 to v31, you will need to upgrade to v31.0.1+ rather than v31.0.0.
Upgrading from v28 to v31
Flows
After upgrading from v28 directly to v31, a "Document Processing (V31)" flow will be created for you, which will reflect your use of Hyperscience in v28. It will also contain your v28 input connections.
If you set up output connections in v28, a "Document Processing Notifications" flow will be created for you, as well. It will contain the output connections you were using in v28. If you did not use any output connections in v28, you will not see this flow in your application.
Upgrading from v28.0.x to v28.2.x
Customers upgrading from v28.0.x to v28.2.0-v28.2.2 were not able to do so if their instances had scheduled trainer tasks for recalibration, finetuning, or auto-thresholding. To avoid this issue, if you are upgrading from v28.0.x to v28.2.x, upgrade to v28.2.3 or later.
Upgrading from v28.1.x or earlier to v30+
Layout variations
With the introduction of Layout Management in v28.2.0, all layouts will have at least one variation, and Structured layouts can have multiple variations. Upon upgrading to v30 or later, each of your layouts will be converted to a layout with a single layout variation. The system does not do any merging of similar layouts or layout variations, and there are no tools in the application to do so. However, we do offer a script to group similar Structured layout variations into a single Structured layout. Running this script is not required, but it does allow you to take advantage of our Layout Management features. It can be run anytime after upgrading to v30 or later.
To learn more about Layout Management and the script, see the “Layout Management Overview” article for your version of Hyperscience ( v35 | v36 | v37 | v38 | v39 | v40 ). For more information about organizing your layouts, or for assistance in running the script, reach out to your Hyperscience representative.
Upgrading to v27.0.8+, v28.0.10+, v28.2.3+, or v30+
In v27.0.8+, v28.0.10+, v28.2.3+, and v30+, we've included two additional security measures that affect authentication.
Single authentication method
We are enforcing the use of a single authentication method per Hyperscience instance and will invalidate tokens for users created with other methods.
You can determine which authentication method or methods you have enabled by checking your instance’s “.env” file for variables named HS_LOGIN_ENABLE_[NAME_OF_METHOD].
If you don’t see a variable with this kind of name, you have local authentication enabled.
If you do see one or more of those variables, they indicate which methods are enabled in your account.
If you have multiple methods enabled, you must choose one to be your system’s authentication method.
To prevent users on different authentication methods from being logged out upon upgrade, we recommend running the following script to migrate all users to your single authentication method before upgrading:
UPDATE user_profile_userprofile SET auth_method = <AUTH_METHOD> ;In this script, replace <AUTH_METHOD> with the number assigned to your authentication method:
Local - 1
LDAP - 2
SAML - 3
OpenID - 4
If you are upgrading to v28.0.11+, v28.2.4+, or v30+, you do not need to run this script, as the system runs it automatically during the upgrade process.
Automatic token invalidation (optional)
You can also choose to automatically invalidate API tokens for users created through an external authentication method. If you choose to enable automatic token invalidation, the system will invalidate API tokens for users every 12 hours.
If you enable this feature, you may need to add a list of your API-only users to your instance’s “.env” file to exempt those users from token invalidation. If you have application users who need continuous access to the API without logging in to the application, you should include those users in your list, as well. For more information, see External Authentication Methods and API Users.
As part of your upgrade, you will need to complete the following steps to prevent token invalidation for the users you specify:
Verify authentication methods for all of your accounts — Prior to upgrading, check the authentication methods for your accounts to determine whether you need to add them to your list of exempted users. If they are using local authentication, follow the steps in Transitioning local users to external authentication to prevent interruptions to their access.
Add the TOKEN_REVALIDATION_EXEMPTED_USERS variable to your “.env” file — During the upgrade, add this variable as described in External Authentication Methods and API Users. The value of TOKEN_REVALIDATION_EXEMPTED_USERS serves as your list of exempted users.
Complete a one-time authentication — After upgrading and creating your list of exempted users, you will need to reauthenticate the accounts in the list by logging them into the application via your external authentication method.
Transitioning local users to external authentication
If you're using an external authentication method but also have local users that you want to exempt from token invalidation, you will need to:
Re-create these users with your external authentication method.
Add them to the list of exempted users as part of step 2 above.
Log each of these users into the application via your external authentication method as part of step 3 above.
Upgrading to v28.x.x
Submission deadlines
If you are upgrading from a version of Hyperscience that had "High," "Default," and "Low" priorities for submissions, we will assign submission deadlines as described in the sections below.
To learn more about submission-processing deadlines, see the “Prioritizing Submissions” article for your version of Hyperscience ( v35 | v36 | v37 | v38 | v39 | v40 ).
Priorities set with the “priority” field in the API
"High"-priority submissions will have a deadline of 1 hour from submission creation.
"Default"-priority submissions will be assigned the system-default deadline.
"Low"-priority submissions will be given a deadline twice as long after submission-creation time as the system-default deadline.
For more information about the “priority” field, see our API documentation.
Priorities set for specific layouts
"High"-priority submissions will have a deadline of 1 hour from submission creation.
"Default"-priority submissions will have a deadline of system default from submission creation.
"Low"-priority submissions will have a deadline of 48 hours from submission creation.
Priorities set for specific connectors
Priority > 1250: Deadline will be 1 hour after submission creation.
Priority < 750: Deadline will be 48 hours after submission creation.
All other values: Deadline will be set to the default deadline.
Known Issues
The sections below describe the known upgrade issues with specific versions of our application, along with ways to avoid them.
Preparing to upgrade from v28.x.x
Recalibration
Before upgrading to v30, first, upgrade your application to either v28.0.9+ or v28.2.2+. In v28.0.0-28.0.7 and v28.2.0 of our application, an issue in the recalibration component will cause your models to underperform and your automation to be reduced. If you train models with the v30 trainer on v28.0.9+ or v28.2.2+ of the application, you will avoid this drop in automation in v30.
A fix for the recalibration issue was also included in v28.0.8 and v28.2.1, but due to other issues in those versions, you should not use them.
Upgrading from v27.x.x to v28.x.x
Classification models
When you train Classification models on a v28.x.x trainer attached to a v27.x.x application, your application will crash. Therefore, you will need to train your Classification model after upgrading to v28.x.x of the application.