Configuration

Overview

Each Redis service is configured with the below basic structure:

{
    // Common service configuration
    "maximum-outstanding": <maximum outstanding messages>,
    "connection": { 
        // Redis database connection configuration 
    },
    "endpoints": [
        // One endpoint per queue (both read and write)
    ]
}

Common service configuration items are available for all JSLEE services. The configuration items specific to the Redis service are listed below.

Field Type Required? Default Description
maximum-outstanding Integer No 50 The total amount of outstanding messages allowed to processing services. New messages will not be requested from the Redis server for any queue if this limit is reached.
connection Object Yes - The Redis database connection configuration.
endpoints Array No (empty) The list of message queues to manage on the Redis server.

Connection Configuration

The following connection configuration options are available for Redis connectivity.

Field Type Required? Default Description
maxWaitingHandlers Integer No 2048 The maximum allowable number of outstanding requests to Redis. This determines the outstanding queue of responses the client may be waiting for, per service instance.
maxNestedArrays Integer No 32 The maximum number of nested arrays that are returned from Redis. Note that the number of nested arrays in results affects parsing performance; if nested arrays are not used, this value can be low.
select Integer No null The ID of the database to connect to when connecting to the Redis server.
type String No "STANDALONE" The type of connection to use, either CLUSTER, SENTINEL, or STANDALONE. This should be set to align with your Redis configuration. Note that a Redis Enterprise cluster connection should use STANDALONE.
useSlave String No "NEVER" Whether to use Redis slaves when running on master nodes. This may be ALWAYS to exclusively use slaves, NEVER to exclusively use masters, or SHARE to use both.
tcp Object No (various) Common TCP/IP connectivity options to use for all TCP/IP connections made to the Redis server.
password String No null The password to use to log in to the Redis server, if required. If not specified, then no password is used to authenticate.
endpoint String No redis://127.0.0.1:6379 The Redis server to connect to, as a colon-separated TCP/IP host and port. The default value is only used if endpoints is not provided.
endpoints Array No null A list of endpoints to connect to first, to reach a clustered or sentinel Redis instance. Each endpoint must be in the same format as given in endpoint.
password String No - The password to use for the Redis server, if required.

Cluster vs Standalone Connections

The type configuration for the Redis connection determines how the JSLEE manages the Redis connection. The connection type should be set to:

  1. STANDALONE should be used when the JSLEE should connect to a single Redis server only. This covers normal unclustered Redis connections, and also Redis Enterprise clusters which provide a single endpoint to manage the Redis connection (effectively hiding the cluster configuration behind a proxy).

  2. CLUSTER should be used only when the JSLEE is to connect to multiple Redis servers which are themselves clustered. This option should be used only when an Open Source Redis cluster is being used.

Endpoints List

When connecting to a Redis cluster, the cluster’s hostname/IP and port can be given using either endpoint or endpoints. The JSLEE Redis cluster connection manager will automatically connect to the endpoints configured, and then use the CLUSTER NODES Redis command to identify any additional nodes.

The Redis service uses the CLUSTER SLOTS Redis command to determine which redis nodes hold which hash slots.

This allows the configuration to maintain a subset of node details for the cluster, rather than require the configuration to define all nodes explicitly.

The Redis cluster connections will be automatically maintained where possible, however the JSLEE will not automatically pick up new nodes. To enable new nodes, send a configuration reload to the Redis service via the JSLEE console.

Endpoints Configuration

The endpoints list is a list of logical message queues for the Redis service to interact with. Each endpoint queue is configured with a queue name and processing configuration:

{
    "name": "myqueue",
    "enabled": true,
    "source-list": "<source-list-x>",
    "sink-list": "<processed-list-x>",
    "error-list": "<error-list-x>",
    "retry-list": "<retry-list-x>",
    "processing-address": "<jslee-service>",
    "processing-list": "<processing-list-x>",
    "expect-response-class-codes": true
}

If the source-list parameter is defined, the processing-address parameter creates a dependency for the destination value - the endpoint will not poll for list data unless at least one instance of this address is available.

The following configuration options are available for each configured queue:

Field Type Required? Default Description
name String No (source/sink name) The endpoint name that can be referred to by other JSLEE services.
enabled Boolean No true Whether this queue connection to Redis should be enabled or not. Takes effect at service startup and after config reload, but can be overridden by the Endpoint command through the JSLEE console.
source-list String No - The name of the Redis queue for the endpoint to read from and write to. The queue need not exist prior to use. If using a Redis cluster, this must be aligned with the processing-list queue name.
processing-list String Conditional - The name of the Redis queue for the endpoint to use during processing. Redis executes BRPOPLPUSH <source-list> <processing-list>. Required ifsource-list is provided. If using a Redis cluster, this must be aligned with the source-list queue name.
processing-address String Conditional - The address to use when sending all messages read from this queue onward. Required ifsource-list is provided.
processing-key String No - An RFC 6901 document path to be appended to EDR slee-session-id values. If not specified or if the value is unable to be read from this path for any reason, no appending will be done. Note that this requires that the information read from Redis is JSON.
sink-list String Conditional - The name of the Redis list to transfer successfully processed messages to. Must be configured if delete-after-success is false (the default).
error-list Strung Conditional - The name of the Redis list to transfer messages to when the processing endpoint responds with an error. Must be configured if delete-on-error is false (the default).
retry-list String No - The name of the Redis list to transfer messages to when a transient error of some form is received. This includes timeouts attempting to process the message, failures sending messages to the processing address (e.g. because it is disabled), and (if enabled using expect-response-class-codes) error codes received from the processing address that indicate a transient failure. If a transient failure code is received but the retry-list is not listed, the error-list is used instead.
delete-after-success Boolean No false If true, delete a successfully transferred message, rather than transferring to sink-list.
delete-after-error Boolean No false If true, delete the message from processing-list when an error is received from the processing-endpoint (either due to delivery or processing error), rather than transferring to error-list.
expect-response-class-codes Boolean No true The Redis queue can expect ResponseClass error codes from the processing address, allowing the service to distinguish between permanent and temporary errors. The processing service must consistently reply with ResponseClass codes for this to be enabled.

Redis Cluster Key Alignment

When using a source-list pushing to a processing-list in a Redis cluster, the specified values must include a Redis hash tag signifier, defined within curly braces. This value must be the same between the two keys, as the BRPOPLPUSH operation used by Redis contains two actions but is atomic and must therefore operate on the same cluster host.

A sample configuration might be:

        "endpoints": [
          {
            "name": "mno1",
            "source-list": "{mno1}",
            "error-list": "mno1-error",
            "delete-after-success": true,
            "processing-address": "smsgw",
            "processing-list": "{mno1}-processing"
          }

This ensures that the string mno1 is used as the lookup value for key insertion for both lists, allowing the atomic operation to succeed on a single host.

This is not required for the error-list, sink-list, or retry-list keys, as these are only used by single-action operations.

Temporary vs. Permanent Errors

The Redis queue management is sensitive to the form of each failure that occurs when it passes a request message to the processing-address. The following response handling logic is employed by the Redis service:

  1. Any successful response from the processing service is considered a success, and the message is either removed or added to the sink-list if defined.
  2. In cases where a timeout occurs, or there is a failure to reach the service defined by the processing-address, the message will be transferred to the retry-list if one is defined, otherwise the message will be transferred to the error-list (or deleted).
  3. If expect-response-class-codes is false, all other error responses are considered permanent errors, and the message will be transferred to the error-list (or deleted).
  4. If expect-response-class-codes is true, and is not a timeout or service connection error, the error received will be processed:
    1. If the error received can be considered a ResponseClass error code, the code is analysed.
    2. If the error code is a transient (temporary) error code, the message will be transferred to the retry-list, if one is defined, otherwise it will be transferred to the error-list (or deleted).
    3. If the error code is a permanent error code, the message will be transferred to the error-list (or deleted).
    4. If the error code is missing, or cannot be processed as a ResponseCode error code, a log message is printed, and the message is transferred to the error-list (or deleted).

EDRs

EDRs are generated for messages received and sent using the reliable queuing mechanism, but not for key/value storage. The JSLEE EDR subsystem is used for this purpose and EDRs will be generated if the service configuration includes an appropriate edr subsection.

Refer to the Redis service EDR section for information on the fields generated in Redis EDRs.

Statistics

The Redis verticle publishes statistics using built in JSLEE statistics publishing system. Refer to the Redis Statistics for a full list of statistics generated by the Redis service.

Connection Encryption

The JSLEE Redis connection does not support TLS, SSL or other on-the-wire encryption methods natively. For further information on encrypting traffic between the JSLEE and Redis, please review the documentation provided by Redis on the Redis security page.

As noted by this documentation, spiped is recommended for on-the-wire encryption. For connection security, the Redis server password can be set in the JSLEE password field in the connection section of the service configuration:

{
  "connection": {
    "password": "mypassword",
    // ...
  },
}