S3 Cloud Storage setup
Defining S3 Lifecycle configuration rules
To prevent data loss and damage to your instance, follow these guidelines when creating a lifecycle configuration for your file store's S3 bucket:
Do not use the Expire current version of objects action.
You can use either or both of the following actions:
Permanently delete previous versions of objects
Delete expired delete markers or incomplete multipart uploads
For information on how to delete data safely, see the "PII Data Deletion" articles for your version of Hyperscience ( v35 | v36 | v37 | v38 | v39 | v40 ).
To ensure system stability, we recommend that you do not edit the value of any of variables listed on this page after the initial deployment of your system. If you need to do so, contact your Hyperscience representative before making any changes.
The file storage can be configured to work with Amazon S3. The S3 storage mode and S3 bucket name will need to be specified in the “.env” file:
FORMS_STORAGE_MODE=<S3 OR S3_EX OR S3_LEGACY>
FILE_STORE_S3_BUCKET=<bucket_name>Value of FORMS_STORAGE_MODE
In v36.0.6+ and v37 and later, file stores have a six-level directory structure. This structure minimizes the number of files in each directory, allowing for faster data retrieval and improving performance in high-volume instances.
The value of FORMS_STORAGE_MODE depends on the version of Hyperscience you're running:
v36.0.6 + or v37 or later: S3_EX
v36.0.5 or earlier: S3
If you are upgrading to v36.0.6+ or v37 or later, your file store will use the optimized directory structure by default. If you want to continue using the older directory structure, set FORMS_STORAGE_MODE to S3_LEGACY.
All customers with S3 file stores should also specify the Region of their S3 bucket by including the FILE_STORE_S3_REGION variable below (required in v28.0.4 and later). See Amazon's documentation for a list of valid Regions.
FILE_STORE_S3_REGION=<region_name>If you choose to store files in a specific folder rather than at the bucket's root level, you'll need to enter the folder's path in the ".env" file, as well:
FILE_STORE_S3_KEY_PREFIX=</parent_folder_name/folder_name/>With the release of version 27, changes have been made to the required permissions for the S3 bucket. The new IAM policy should look like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:ListBucketMultipartUploads",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::EXAMPLE-BUCKET-NAME"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::EXAMPLE-BUCKET-NAME/*"
}
]
}Optionally, if you use an S3-like internal setup with an alternative URL (that does not point to s3.amazonaws.com), you can specify that URL using the following parameter:
# optionally, override S3 endpoint URL
FILE_STORE_S3_ENDPOINT_URL=<alternative S3 URL>Amazon S3 authentication configuration
If using Amazon S3 as a file storage option, a submission retrieval option, or both, S3 credentials need to be configured. The recommended approach is to configure your EC2 instance to have access to S3 using IAM Roles instead of using explicit credentials.
If IAM Roles are unavailable, you can specify credentials in the ".env" file.
Specifying Amazon S3 credentials (optional)
If a single S3 account is used, specify the following in the ".env" file:
AWS_ACCESS_KEY_ID=<access key>
AWS_SECRET_ACCESS_KEY=<secret access key>If separate S3 accounts are used for the file storage and submission retrieval options, specify the following in the ".env" file:
FILE_STORE_S3_ACCESS_ID=<access key>
FILE_STORE_S3_ACCESS_KEY=<secret access key>
SUBMISSION_RETRIEVAL_STORE_S3_ACCESS_ID=<access key>
SUBMISSION_RETRIEVAL_STORE_S3_ACCESS_KEY=<secret access key> In v32 and later, you can choose to store your system-level credentials in a secrets manager. To learn more about our secrets-management integration options, see Secrets Management.