Message Queue Notifier

  • Published on Oct 10, 2024
  • 6 minute(s) read

With the Message Queue Notifier Output Block, you can configure connections to the following message queue types:

  • ActiveMQ

  • Amazon SQS

  • IBM MQ

  • RabbitMQ

To select the message queue type you want to use for a Message Queue Notifier connection, click on its name in the Message Queue Type drop-down list at the top of the block’s settings.

If you would like to use a Message Queue Notifier connection in a SaaS instance and your MQ service is not public, you should create a firewall pinhole for the connection. Contact your Hyperscience representative for more information.

Sample use cases

  • I have extracted content from insurance forms and want to send the JSON downstream to my IBM MQ in order to populate other systems.

  • I have transformed submission data and want to send it to my RabbitMQ, where it will be picked up by internal software to be further processed.

Block settings tables

In addition to the settings outlined below, you can also configure the settings described in Universal Integration Block Settings. 

ActiveMQ

Name

Required?

Description

Username

Yes

The username of a user that has access to the message queue.

Password

Yes

The password of a user that has access to the message queue.

To edit the password, click Edit value, modify the password, and then click Done.

Queue Name

Yes

The name of the message queue in ActiveMQ.

Host Name

Yes

The hostname of the system where the message queue is being run.

Amazon SQS

Name

Required?

Description

AWS Region

Yes

The AWS region of the data's destination.

Access Key ID

No

The access key ID allows access to the message queue.

This setting is only available if Use AWS EC2 Instance IAM Role Credentials is not selected.

Secret Access Key

No

The secret access key allows access to the message queue.

To edit the key, click Edit value, modify the key, and then click Done.

This setting is only available if Use AWS EC2 Instance IAM Role Credentials is not selected.

Queue URL

Yes

The queue URL for receiving messages.

Group ID for FIFO Queues

Yes

The group ID will be added to messages, which ensures that AWS processes messages in the correct order. 

Use AWS EC2 Instance IAM Role Credentials 

Yes

If selected, credentials are obtained from the EC2 instance directly, and the Access Key ID and Secret Key are not present.

Enabled by default.

Additional SQS metadata

No

Additional metadata that you'd like to send to Amazon SQS.

Additional configurations

The following permissions should be granted via the AWS interface:

  • sqs:GetQueueUrl

  • sqs:SendMessage

Message size

Note that Amazon imposes certain limitations on the size of messages sent to Amazon SQS message queues as documented in their Developer Guide. At the time of writing this article, the maximum message size allowed is 256KB. Depending on the specifics of your use case, this limit may prevent you from using the Amazon SQS connection effectively.

IBM MQ

Name

Required?

Description

No Auth Credentials Required

Yes

Indicates whether the connection requires a username and password.

If selected, Username and Password are not present.

Username

Yes, if available

The username of a user that has access to the queue manager.

This field is only available if No Auth Credentials Required is not selected.

Password

Yes, if available

The password of a user that has access to the queue manager.

To edit the password, click Edit value, modify the password, and then click Done.

This field is only available if No Auth Credentials Required is not selected.

Queue Name

Yes

The name of the message queue in IBM MQ.

Host Name

Yes

The hostname of the system where the queue manager is being run.

Port Number

Yes

The port number the connection should use to connect to the queue manager.

Defaults to 1414.

Queue Manager 

Yes

The name of the message queue's queue manager in IBM MQ.

Channel

Yes

The queue manager's channel in IBM MQ.

SSL Cipher Suite

Yes, if you want to use an SSL connection

The CipherSuite connection should use to communicate with the queue manager. 

If you are not using an SSL connection to connect to the queue manager, select None in this field.

RabbitMQ

Name

Required?

Description

Username

Yes

The username of a user that has access to the message queue.

Password

Yes

The password of a user that has access to the message queue.

To edit the password, click Edit value, modify the password, and then click Done.

Queue Name

Yes

The name of the message queue in IBM MQ.

Host Name

Yes

The hostname of the system where the message queue is being run.

Port Number

Yes

The port number the connection should use to connect to the message queue.

Defaults to 5672.

Connection Type

Yes

Specify whether the connection is a TCP or SSL connection.

Virtual Host

Yes

The name of the virtual host where the message queue is being run.

Exchange

Yes

The exchange that the connection should send messages to.

Routing Key

Yes

The routing key that the connection should add to messages before sending them to RabbitMQ.

Setting up Message Queue Notifier

Initial setup for all Message Queue Notifiers

These steps apply to all supported Message Queue Notifier Blocks. 

  1. Log in to your Hyperscience instance.

  2. Navigate to Flows and choose any flow.

  3. Scroll to the end of the flow in the Flow Studio and click Outputs.

  4. Click Add and select Message Queue Notifier Output Block from the list.

  5. Click the Add Connection button.

  6. Configure the payload options – API Version, Export Type, and Routing Filter.

  7. From the Message Queue Type drop-down menu, select your Message Queue Type.

  8. Follow the additional configuration steps outlined for your Message Queue Type in the Connection-specific configurations section below.

Connection-specific configurations

In addition to the initial setup steps outlined above, each type of Message Queue Notifier has its own specific setup steps, which are described in the sections below.

ActiveMQ

  1. Within ActiveMQ, whether it is configured in Amazon or elsewhere, find your username and password. Enter the username and password into the corresponding Username and Password fields in the Message Queue Notifier Block.

  2. Find your queue name. Input that into the Queue Name field.

  3. Finally, input your hostname into the corresponding field.

Amazon SQS

  1. Navigate to your Amazon Simple Queue Service.

  2. Choose or create the relevant input queue. We support Standard and FIFO Queues.

  3. In the configuration of the Message Queue Notifier Block, you can configure delays, timeouts, retention, and more.

  4. If you are using a FIFO Queue, enter the Group ID in the Group ID for the FIFO Queues field.

  5. We provide maximum flexibility in authentication and access. As long as Hyperscience has access to the queue, this integration should automatically work.

Additional configurations

The following permissions should be granted via the AWS interface:

  • sqs:GetQueueUrl

  • sqs:SendMessage

Message size

Note that Amazon imposes certain limitations on the size of messages sent to Amazon SQS message queues as documented in Amazon’s Amazon SQS quotas. Depending on the specifics of your use case, this limit may prevent you from using the Amazon SQS connection effectively.

For more information on the structure of the JSON object that should be passed to the message queue in order for Hyperscience to read the appropriate input image files, refer to the Submission Creation section of our API documentation.

RabbitMQ

  1. Add a new queue in RabbitMQ. Our Message Queue Notifier Block is very flexible and can support any number of RabbitMQ configurations. We recommend setting up this connection in the most convenient way for your processes.

  2. Once you have set up the queue, fill out the corresponding fields in the Message Queue Notifier Block.

  3. For relevant documentation, see RabbitMQ’s Queues documentation.

IBM MQ

  1. Create or find the relevant input queue within IBM MQ. We recommend using a Local or Remote queue for ease of use. If you are comfortable with Model or Alias queues, please feel free to use those as well.

  2. Within the Message Queue Notifier Block, select whether or not Auth Credentials are required for your queue. If they are required, fill out the credentials.

  3. Confirm whether your queue is configured to receive a cipher suite. If it is not, make sure to set it in the SSL Cipher Suite field. Without this filled out, your integration could fail.