Redis Service
Introduction
The Redis N2JSLEE service provides Redis integration as a service in two forms:
- For key/value storage, the Redis service provides a convenient solution for storage, without the need for other
services to access a Redis instance directly. Its general Redis interface supports a wide variety of non-blocking
Redis requests, including
SET
,GET
,UNLINK
,LPUSH
,LPOP
, etc. - For reliable message queues, the Redis service provides Redis List access in the manner described by the Redis command RPOPLPUSH. When configured with one or more queues, the Redis service can be used as a streaming message service with both source and sink endpoints.
Each of these operations may produce EDRs as part of message processing.
Each configured Redis service may be connected to at most one Redis database, either in Sentinel or Cluster mode. Once connected, the Redis database is available for storage by other N2JSLEE services.
Note that Redis Streams and the Redis Pub/Sub model are not supported by this service.
Queues
Each queue is defined as an endpoint in the Redis service configuration. N2JSLEE management commands may be used to enable, disable and quesce each queue individually. When disabled, the queue is considered unavailable for both read and write, regardless of the underlying Redis connectivity maintained by this service.
Each configured queue monitors a Redis list, <source-list>
, which may be independently
accessed by multiple threads, nodes, and
services (including non-N2JSLEE services). When an external entity pushes a message to <source-list>
the Redis service
will read the message using the BRPOPLPUSH
which will atomically push the message to a secondary queue <processing-list>
as soon as the Redis server receives the message.
The full message read from Redis as part of this command will be transferred to another N2JSLEE service, the Processing
Service. After the Processing Service completes appropriate processing of the received message, it will reply to the
Redis service indicating the success or failure of the message processing. The Redis service will then move the processed
message from <processing-list>
to a final list, either <success-list>
or <error-list>
as appropriate. Alternatively, the
message may be removed from <processing-list>
without being pushed to a final list.
The following sequence diagram summarises this process:
Data Formats
Redis databases may store any arbitrary binary data as keys or values. The Redis interface supports any form of data supported by the underlying Vert.X Response object.
In addition to the general data formats, the Redis interface also supports receiving a JsonSchemaDocumentEvent
which
is automatically converted into a JSON string for storage.
Sending Requests
The Redis interface will accept JSLEERedisRequest
and JsonSchemaDocumentEvent
requests at the general service’s
endpoint name (i.e the configured name of the service). It is also possible to submit to specific Redis lists by
sending a JsonSchemaDocumentEvent
same request to a queue endpoint, as long as that endpoint has a <sink-list>
defined.
Cluster Management
When connecting to a Redis cluster, a connection will be maintained to each Redis cluster node, as determined by the
CLUSTER NODES
Redis command. Service configuration need only provide at least one (preferably two) endpoints to
connect to achieve full cluster connectivity.
However, the JSLEE does not automatically poll the Redis cluster for changes. The JSLEE should be informed of changes to cluster layout, including new nodes, new masters (or replicas), or changes to hash slot balance across master nodes by sending a configuration reload request to the Redis endpoint(s) of the JSLEE.
To add a new master node, or replica node, to the cluster:
- Perform the Redis configuration to add the new node(s) to the cluster (e.g. using
redis-cli --cluster add-node
) - Do not rebalance cluster hash slots to the new nodes yet.
- Use the JSLEE console to send a
service reload <redis-service>
command. - Monitor the JSLEE log to ensure the new Redis node(s) are connected.
- Rebalance hash slots to the new master as required.
- Use the JSLEE console to send a
service reload <redis-service>
command to force the JSLEE’s hash slot cache to be updated.
Similar, a managed disconnect of one or nodes should be done by rebalancing hash slots to the nodes to be kept, requesting the configuration reload of the JSLEE via the console, and only then shutting down the Redis nodes.