Redis Service

Introduction

The Redis N2JSLEE service provides Redis integration as a service in two forms:

  1. For key/value storage, the Redis service provides a convenient solution for storage, without the need for other services to access a Redis instance directly. Its general Redis interface supports a wide variety of non-blocking Redis requests, including SET, GET, UNLINK, LPUSH, LPOP, etc.
  2. For reliable message queues, the Redis service provides Redis List access in the manner described by the Redis command RPOPLPUSH. When configured with one or more queues, the Redis service can be used as a streaming message service with both source and sink endpoints.

Each of these operations may produce EDRs as part of message processing.

Each configured Redis service may be connected to at most one Redis database, either in Sentinel or Cluster mode. Once connected, the Redis database is available for storage by other N2JSLEE services.

Note that Redis Streams and the Redis Pub/Sub model are not supported by this service.

Queues

Each queue is defined as an endpoint in the Redis service configuration. N2JSLEE management commands may be used to enable, disable and quesce each queue individually. When disabled, the queue is considered unavailable for both read and write, regardless of the underlying Redis connectivity maintained by this service.

Each configured queue monitors a Redis list, <source-list>, which may be independently accessed by multiple threads, nodes, and services (including non-N2JSLEE services). When an external entity pushes a message to <source-list> the Redis service will read the message using the BRPOPLPUSH which will atomically push the message to a secondary queue <processing-list> as soon as the Redis server receives the message.

The full message read from Redis as part of this command will be transferred to another N2JSLEE service, the Processing Service. After the Processing Service completes appropriate processing of the received message, it will reply to the Redis service indicating the success or failure of the message processing. The Redis service will then move the processed message from <processing-list> to a final list, either <success-list> or <error-list> as appropriate. Alternatively, the message may be removed from <processing-list> without being pushed to a final list.

The following sequence diagram summarises this process:

Redis message flow

Data Formats

Redis databases may store any arbitrary binary data as keys or values. The Redis interface supports any form of data supported by the underlying Vert.X Response object.

In addition to the general data formats, the Redis interface also supports receiving a JsonSchemaDocumentEvent which is automatically converted into a JSON string for storage.

Sending Requests

The Redis interface will accept JSLEERedisRequest and JsonSchemaDocumentEvent requests at the general service’s endpoint name (i.e the configured name of the service). It is also possible to submit to specific Redis lists by sending a JsonSchemaDocumentEvent same request to a queue endpoint, as long as that endpoint has a <sink-list> defined.

Cluster Management

When connecting to a Redis cluster, a connection will be maintained to each Redis cluster node, as determined by the CLUSTER NODES Redis command. Service configuration need only provide at least one (preferably two) endpoints to connect to achieve full cluster connectivity.

However, the JSLEE does not automatically poll the Redis cluster for changes. The JSLEE should be informed of changes to cluster layout, including new nodes, new masters (or replicas), or changes to hash slot balance across master nodes by sending a configuration reload request to the Redis endpoint(s) of the JSLEE.

To add a new master node, or replica node, to the cluster:

  1. Perform the Redis configuration to add the new node(s) to the cluster (e.g. using redis-cli --cluster add-node)
  2. Do not rebalance cluster hash slots to the new nodes yet.
  3. Use the JSLEE console to send a service reload <redis-service> command.
  4. Monitor the JSLEE log to ensure the new Redis node(s) are connected.
  5. Rebalance hash slots to the new master as required.
  6. Use the JSLEE console to send a service reload <redis-service> command to force the JSLEE’s hash slot cache to be updated.

Similar, a managed disconnect of one or nodes should be done by rebalancing hash slots to the nodes to be kept, requesting the configuration reload of the JSLEE via the console, and only then shutting down the Redis nodes.