Universal Folder Listener

Overview

The Universal Folder Listener allows you to ingest files from a Folder on your system.

The Universal Folder Listener Input connection is available only for on-premise / private cloud instances of Hyperscience.

Sample use cases

• Another system places documents in a system folder on a regular basis. I want to ingest those files one by one.

• I put several related files into subfolders to be ingested as one submission per subfolder into Hyperscience.

• I want to regularly scan and ingest certain file types from a shared system folder.

Block settings table

In addition to the settings outlined below, you can also configure the settings described in Universal Integration Block Settings. 

Setting up Universal Folder Listener

To set up the Universal Folder Listener, enter the settings as described in the Block settings table above. You do not need to configure anything in the folder you are monitoring.

Migrating from the Folder Listener

There are numerous benefits for users upgrading from the legacy Folder Listener to the Universal Folder Listener. These include:

• Submissions with multiple files

• Simplified file structure

• Support for Cases

• Numerous UX improvements

Before setting up a Universal Folder Listener connector and retiring your Folder Listener connection, you need to complete some preparatory steps.

Re-evaluate your folder structure

As mentioned previously, the Universal Folder Listener accepts both single files and folders of files as submissions. Whether a file is processed as its own submission or as part of a larger submission depends on where it is located in the base folder:

• If it is directly under the base folder, the system processes it as an individual submission.

• If it is in a subfolder of the base folder, the system considers it part of a larger submission, which consists of all files in the subfolder and in any folders within that subfolder.

Depending on how you want your files to be processed, you may need to make some changes to your base folder's structure before using the Universal Folder Listener.

Verify the path of your base folder

The system uses the file path in the FS_INPUT_PATH ".env" file variable as the default location of the base folder. While you can enter any subdirectory of that folder in the Folder To Scan For Submissions field, for greater security, you may want to modify the value of FS_INPUT_PATH if you are certain you want to use one of its subfolders as your base folder.

Update your metadata files

The Folder Listener connector supported text files that contained submission metadata. The Universal Folder Listener can accept a JSON file for each submission that contains metadata, case data, and an external_id. The name of this file depends on whether the submission consists of a single file or a folder of files:

• filename.ext.json for individual files, where filename is the name of the file and ext is the file's extension

• foldername.json for folders of files, where foldername is the name of the base folder's subfolder

The metadata file for a file or folder must be in the base folder. If you put it in one of the base folder's subfolders, a submission won't be created from the file or folder the metadata is for. 

An example JSON file for a Universal Folder Listener submission appears below.

{
    "metadata": {
         "test": "Metadata for inner1"
    },
    "cases": [{
         "external_case_id": "900",
         "filenames": ["div_lic_1.jpg", "div_lic_1.jpg"]
    }],
    "external_id": "123"
}

By contrast, the metadata text files supported by the original Folder Listener (filename_index.txt) consist of key/value pairs that you define:

{
    "key1": "value1",
    "Key2": "value2"
} 

Therefore, when migrating to the Universal Folder Listener, you need to change the names, file types, and content of your metadata files, as outlined in the table below.