EDR File Lua Agent

Introduction

The EdrFileLuaAgent is an synchronous helper for Lua scripts running within the LogicApp. The EDR File Agent may be used by any Lua script, regardless of which Service initiated that script.

The EdrFileLuaAgent monitors EDR files directly on disk, looking for new and modified files. It reads the EDR text lines into an in-memory cache and provides an API for querying those EDRs based on key field values.

The EdrFileLuaAgent is uses specifically for regression testing of Event Data Records generated by applications.

NOTE: Althought the EdrFileLuaAgent module lives within the EdrApp directory, it does not communicate directly with the EdrApp at run-time.

EDR File Lua Agent methods are accessed via the “n2.n2svcd.edr_file_agent” module:

local edr_file_agent = require "n2.n2svcd.edr_file_agent"

Configuring EdrFileLuaAgent

The EdrFileLuaAgent is configured within a LogicApp.

<?xml version="1.0" encoding="utf-8"?>
<n2svcd>
  ...
  <applications>
    ...
    <application name="Logic" module="LogicApp">
      <include>
        <lib>../apps/logic/lib</lib>
      </include>
      <config>
        <services>
          ...
        </services>
        <agents>
          <agent module="EdrApp::EdrFileLuaAgent" libs="../apps/edr/lib">
              <config max_edrs_per_stream="2000" edr_margin_secs="3">
                  <streams>
                      <stream key="n2logic" directory="/tmp/edr"/>
                  </streams>
              </config>
          </agent>
        </agents>
      </config>
    </application>
    ...
  </application>
  ...
</n2svcd>

Under normal installation, the following agent attributes apply:

Parameter Name Type XML Type Description
module EdrApp::EdrFileLuaAgent Attribute [Required] The module name containing the Lua Agent code.
libs ../apps/db/lib Attribute Location of the module for EdrFileLuaAgent.
config Object Element Container for extended configuration for this Application instance.
.max_edrs_per_stream Integer Attribute The maximum number of EDRs to cache for each stream. Excess EDRs are discarded.
(Default = 1000)
.edr_check_ms Integer Attribute How often (in milliseconds) to check the EDR directory for new/modified EDR files and read new records.
(Default = 200)
.edr_margin_secs Integer Attribute A grace margin when filtering for EDRs with a time-stamp that is more recent than "start of call".
(Default = 2)
.edr_retire_secs Integer Attribute After the mtime for EDR file is this many seconds old, it is considered "retired" and the mtime will no longer be checked.
(Default = 300)
.streams Array Element Array of stream elements defining EDR file streams.

EDR File Streams

The EdrApp writes EDRs in “streams” of files which are written to a common directory and which have a common filename prefix.

The EDR File Agent reads one or more of these streams. It does not need to process all streams written by the EdrApp, only those which are of interest to test scripts.

Each stream Object in the config.streams Array is configured as follows.

Parameter Name Type XML Type Description
key String Attribute [Required] The stream key, matching the key from the EDR App Configuration.
directory String Attribute [Required] The full directory path where the EDR stream is being written, as configured in the EDR App Configuration or secondary directory after some post-processing.

The EdrFileLuaAgent API

All methods may raise a Lua Error in the case of exception, including:

Multiple Stream Keys

All methods may accept a single stream key to search, or a comma-separated list of stream keys.

In any case, then the first (or only) stream key in the list is the primary stream key, which will always be used initially to find the primary EDR, along with any other EDRs in that primary stream which are associated with the same instance GID.

Then if additional secondary stream keys are given, then those secondary streams will be searched to find any EDRs which have an upstream GID equal to the instance GID of the primary EDR(s).

Method Results

All methods return a single result with the following structure:

Parameter Type Description
result Table Container for the various returned parameters.
.instance_idx String The instance IDX for the instance whose EDR we matched.
e.g. 6c064234.
.upstream_idxs Table The instance IDXs for the instances upstream of the instance whose EDR we matched.
The first entry is the instance IDX associated with the originating instance.
.instance_gid String The instance GID for the instance whose EDR we matched.
e.g. [ scp01~Logic~1626138304~6c064234 ].
.upstream_gids Table The instance GIDs for the instances that are upstream of the instance whose EDR we matched.
The first entry is the instance GID associated with the originating instance.
.secondary_gids Array of String An array of secondary (downstream) GIDs, one for each additional (secondary) stream key in the original request.
A UNDEF value will be placed in the array if the secondary stream did not match any secondary EDRs.
e.g. [ kirby~Logic~1699132449~42113d92 ].
.secondary_idxs Array of String Array of only the Instance IDX parts of the GIDs in the secondary_gids element described above.
e.g. [ 42113d92 ].
.count Integer The number of EDRs returned.
.list List of Table A structure which holds all returned EDRs.
At the top level, this is a Lua table where the keys are the EDR event Types.
Within each event type table entry, the value is a list of all EDRs of that type.
Within that list each EDR is a table keyed by the EDR field names, plus header fields.

NOTE: The per-EDR tables in .list also include the following special field names which are parsed from the EDR header for each EDR.

.find_fields [Synchronous]

The find_fields method examines its cache of EDRs and begins by attempting to identify the first EDR:

Once that first EDR is identified, it returns all EDRs:

An error is raised in the following cases:

The find_fields method takes the following parameters.

Parameter Type Description
stream_key String [Required] The key of the stream to search, as configured in the EdrFileLuaAgent and EdrApp.
This may be a comma-separated list of stream keys, as described above.
match_event String [Required] The required event type for the primary correlating EDR.
match_fields Table [Required] A table containing one or more key/value pairs to exact-match in an EDR.
start_epoch Positive Integer An epoch value to use as the start time to search from. Note that this is a value without sub-second precision. Consider waiting in scripts in order to avoid multiple instance matches.
(Default: script execution start time)

Example (EDR Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

local arg_EDR_FILE = edr_file_agent.find_fields ('n2scp', 'INITIALDP', { CALLED = '4401234567' })
n2svcd.match_kpath_size (arg_EDR_FILE, 'list.INITIALDP', 1, true)
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].CALLING', '09773322')
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].TRIGGER', 'ORIG')

.confirm_none [Synchronous]

The confirm_none method acts in the opposite manner to the find_fields method by attempting to confirm any of:

In all cases, the “at” time of the EDR is treated identically to the find_fields method.

An error is raised if one or more matching EDRs were found.

All parameters for the confirm_none method are the same as the `find_fields' method.

Example (EDR Negative Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

local arg_EDR_FILE = edr_file_agent.confirm_none ('n2scp', 'INITIALDP', { CALLED = '4401234567' })
n2svcd.match_kpath_exists (arg_EDR_FILE, 'list.INITIALDP', false)

.find_gid [Synchronous]

The find_gid method examines its cache of EDRs and returns all EDRs:

The find_gid method takes the following parameters.

Parameter Type Description
stream_key String [Required] The key of the stream to search, as configured in the EdrFileLuaAgent and EdrApp.
This may be a comma-separated list of stream keys, as described above.
instance_gid String [Required] The required instance GID that identifies the instance whose EDRs should be returned.

Example (EDR Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

arg_EDR_FILE = edr_file_agent.find_gid ('n2scp', 'scp01~LHO SCP~1626138304~6c064234')
n2svcd.match_kpath_size (arg_EDR_FILE, 'list.INITIAL_DP', 1, true)
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].CALLING', '09773322')
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.INITIALDP.[0].TRIGGER', 'ORIG')

.find_upstream_gid [Synchronous]

The find_upstream_gid method examines its cache of EDRs and begins by attempting to identify the first EDR:

Once that first EDR is identified, it return all EDRs:

Note that in general this find_upstream_gid method will not be required, since in most cases it is possible to achieve the same result more simply by specifying a comma-separated list of stream keys to the find_fields or find_gid methods.

An error is raised in the following cases:

The find_upstream_gid method takes the following parameters.

Parameter Type Description
stream_key String [Required] The key of the stream to search, as configured in the EdrFileLuaAgent and EdrApp.
This may be a comma-separated list of stream keys, as described above.
instance_gid String [Required] The instance GID that identifies a required preceding instance for the target EDRs.
app_name String [Required] The name of the application that is required to be associated with the instance for the primary correlating EDR.

Example (EDR Matching):

local n2svcd = require "n2.n2svcd"
local edr_file_agent = require "n2.n2svcd.edr_file_agent"

local rest_args = ...

arg_EDR_FILE = edr_file_agent.find_upstream_gid ('n2scp', 'scp01~LHO SCP~1626138304~6c064234', 'Logic')
n2svcd.match_kpath_size (arg_EDR_FILE, 'list.LOGIC', 1, true)
n2svcd.match_kpath_string (arg_EDR_FILE, 'list.LOGIC.[0].CUSTOM', 'Custom 1')