☰
Navigation :

Common EDR Format

EDR Common Format

All Event Data Records (EDRs) generated by n2svcd (and associated packages such as n2sip, n2scp) have the same basic syntax and record format.

Here is an example complete EDR file containing only one EDR:

#HEADER|FILENAME=/tmp/edr/n2svcd_candyfloss_20210322005441924721_131977.edr|TIME_START=1616374481|HOSTNAME=testscp-01
2021-03-22 00:54:41.919<SCP-DUMMY-1616374153-1893f994>SHUTDOWN|EXCEPTION=Overdue TCAP response for ERBCSM %5B2%5D.
#FOOTER|TIME_FINISH=1616374485|NUM_EDRS=1|NUM_BYTES=233

EDR Lines

Each EDR is a single “line” in the file, terminated by a LF.

There are three special fields at the start of each record, followed by none or more standard fields specific to the event type.

Here is an example EDR line:

2022-09-28 00:06:05.748<hostname.n2.nz~SIGTRAN~1663737098~2b7551ea,hostname.n2.nz~SCP~1663737098~50bbf05e>
INITIALDP|BC=8090a3|CALLED=063581140|CALLING=021682277|CLD=63581140:3:0:1|CLG=21682277:3:0:1:0:3|CPC=0a|ET=3|
HLC=9181|INAP=cs1|LTID=2b7551ea|RGTD=640000001|RSSN=12|RTID=7880daaa|SK=123|TRIGGER=ORIG

Note that extra line breaks have been inserted for readability. In practice, this is all one single line.

In this case the special fields are:

Special Fields

Event Log Timestamp Field

This is the time at which the originating application determined that this EDR should be generated.

This is in UTC (aka GMT), and is in YYYY-MM-DD HH:MM:SS.sss format.

The EDR is encoded, and sent through the shared message bus to the EdrApp, where it might also spend some time in-memory before being written to disk. So the actual write-to-disk time will naturally be slightly after this timestamp.

Conversely, this actual inbound message or event which caused this EDR will of course have happened slightly prior to this time. There will then be additional processing, including perhaps confirmation delay. Hence, when interpreting this timestamp, it is important to consider “at what point in the overall processing” was this EDR generated.

Key Field

This field contains the keys for a specific “instance” across one or more specific applications, as experienced up until the time the EDR was written.

The first application to process a given traffic instance will be the initial key field in the array. If control of the instance is given to another EDR-producing application, any EDRs generated there will include both the first application’s key field and the new key field for the second application. There may therefore be 1..N key fields in any given EDR’s key array.

Each key field is constructed from four components, separated by a tilde ~ character.

  1. The Hostname of the serving node.
  2. The Application Name that generated the EDR.
  3. The Application Start Epoch for the application.
  4. The Instance IDX.

The Hostname is the configured name of the host where the N2SVCD service is running.

The Application Name is the n2svcd application name as configured in the n2svcd.xml service daemon configuration file. It consists of a name part (alphanumeric, space, underscore, hyphen), with an optional (colon + integer) “repeat number”, depending on the configuration.

The application name will match the POSIX regular expression: /^[a-zA-Z0-9 _\-]+(:[0-9]+)?$/

An application will have the :[0-9]+ suffix when it has been “repeated” for performance reasons, i.e. multiple copies of that application are being run, each in a separate process.

The Application Start Epoch is the integer UTC epoch time at which this application was last started. If an application dies and is restarted by the WatchdogApp then it will have a new epoch time, ensuring that the Instance IDX is still unique in combination.

The Instance IDX is an automatically-assigned hex value for each processing context managed by an application. An instance may be a voice call, a REST request/response pair, a start-to-finish execution of a LUA script, etc.

The instance IDX will match the POSIX regular expression: /^[0-9a-f]{8}$/

When an application starts up, it chooses a random starting point for the Instance IDX field, and then increments by one for each new call instance (rotating if it reaches ffffffff).

There is of course a small chance that one application will choose an Instance IDX which overlaps with the values chosen randomly by one of the other applications. Equally, there is a small chance that an application (after dying and being restarted) will choose an Instance IDX which overlaps with the values used by its previous incarnation.

However, the combined four-part field with Name, Start Epoch, and IDX is guaranteed to be completely unique.

Event Type Field

The event type is a string which indicates the nature of the event which is being recorded.

The event type will match the POSIX regular expression: /^[a-zA-Z][a-zA-Z0-9_\-]*$/

The event type then determines what additional standard fields may then follow. Refer to the appropriate documentation for each application that generates EDRs.

Standard Fields

Each standard field is separated by a vertical pipe | character.

Each standard field consists of a field name, then an equals = character, then a field simple scalar value or list of simple scalar values.

The field name must match the POSIX regular expression: /^[a-zA-Z][a-zA-Z0-9_\-]*$/

Standard Field Values (Scalar)

A scalar field value is represented in ASCII, with some characters “escaped” using the % percent-encoding syntax which is standard for Uniform Resource Identifiers (URI) on the Internet.

Safe characters must match this POSIX regular expression: /A-Za-z0-9\-\._~:; \/@/

This is similar to RFC 3986, except that extra characters are considered “safe” in EDR lines, which are not strictly “safe” in RFC 3986. This is a pragmatic decision made for readability, since these characters are quite common in EDR records for some key applications. These additional fields are:

(:) COLON
(;) SEMICOLON
( ) SPACE
(/) SLASH
(@) AT SIGN

All non-safe characters will be replaced with % specifying the ASCII character code. For example, the equals character = in an EDR field value will be represented by %3D. The percent character % itself is represented by %25.

An “undefined” value is represented by the empty string. When reading an EDR file, there is no way to distinguish between an EDR field with zero-length (empty) string value and one with an undefined value.

Standard Field Values (List)

A standard field value may consiste of a “list” of values.

In this case, an unescaped comma character , separator is used. For example, a list of played announcement IDs with field name PID and two list entries 300 and 301 would be represented in the EDR file as:

PID=300,301

A list of scalars is the most complex format supported by the EDR. Nested sub-fields are not supported.

Every EDR file begins with a #HEADER line.

#HEADER|FILENAME=(full file path)|TIME_START=(epoch)|HOSTNAME=(hostname)

This consists of the string #HEADER, followed by three header fields. These are also separated by a vertical pipe | character, and have name and value parts separated by an equals = character.

The header field names are:

Every well-formed EDR file begins with a #FOOTER line.

#FOOTER|TIME_FINISH=1616374485|NUM_EDRS=1|NUM_BYTES=233

This consists of the string #FOOTER, followed by three header fields. These are also separated by a vertical pipe | character, and have name and value parts separated by an equals = character.

The footer field names are:

The NUM_EDRS includes only the EDR lines. The #HEADER and #FOOTER lines are not included in this count.

The NUM_BYTES includes the bytes for the #HEADER line, and all of the bytes for the EDR lines, including the one-byte for the LF (line feed) character written for the header and EDR lines. It does not include the bytes for the #FOOTER itself.