Lua Test

Overview

The lua_test binary program allows users to execute one or more Lua-defined test scripts against a running N2SVCD RestServerApp application (supported by a LogicApp application). This program uses the StartRun REST API endpoint to begin executing each test run, and then uses the QueryRun REST API endpoint to periodically check each test run until it completes.

This program may be used to either:

  1. A single test run, which executes a single instance.
  2. A single test run, which executes multiple instances (specify --total greater than one).
  3. Multiple test runs, each of which run in sequence (specify more than one .lua file).
  4. Multiple test runs, running in parallel (specify more than one .lua file, and specify --parallel greater than one).

The default installation location for this binary is /usr/share/n2svcd/bin/lua_test.

Running lua_test --help will show the program usage.

usage: lua_test <lua-file> [... <lua-file>]
        --host <rest-svc-host>          Default = 'localhost'
        --port <rest-svc-port>          Default = 9010
        --quiet                         (suppress progress noise)
        --minimal                       (suppress even more progress noise)
        --space                         (output a newline between tests)
        --failure-only                  (when running multiple test scripts only print failed tests in summary)
        --summary-totals                (when running multiple test scripts print totals in summary)
        --parallel <run-count>          The maximum number of test runs to start in parallel
        --google-test-output <out-file> Output a Google test formatted XML file that can be used by invoking applications to display run statistics.

        --trace_level <0|1|2|3>         (request n2svcd to return debug traces)
        --trace <0|1|2|3>               (request n2svcd to return debug traces)
        --debug                         (--trace_level = 1)
        --dump                          (--trace_level = 2)
        --spam                          (--trace_level = 3)
        --load-details                  (track test details under load)

        --total <total>                 (requires --cps and/or --backlog)
        --ramp <ramp-seconds>           (requires --cps and/or --backlog)
        --cps <cps>
        --backlog <backlog>

        --variables <variables>         A JSON string containing arbitrary variables to be used in the Lua script.
        --variables-file <filename>     A filename for a file containing arbitrary JSON variables to be used in the Lua script.

        --dpc <point-code>              Set variable {scp}{dpc}
        --dssn <subsystem-number>       Set variable {scp}{dssn}
        --dgt_digits <digits>           Set variable {scp}{dgt_digits}
        --dgt_noa <nature-of-address>   Set variable {scp}{dgt_noa}
        --dgt_np <numbering-plan>       Set variable {scp}{dgt_np}
        --dgt_tt <translation-type>     Set variable {scp}{dgt_tt}

        --smpp <agent-route>            Set variable {route}{smpp}
        --pi <agent-route>              Set variable {route}{pi}
        --soap <agent-route>            Set variable {route}{soap}
        --diameter <agent-route>        Set variable {route}{diameter} 
        --sigtran <agent-route>         Set variable {route}{sigtran}
        --rest <agent-route>            Set variable {route}{rest}

Usage

lua_test can be used to run a single Lua test definition, in which case a single test summary will be output.

./lua_test ../test/regression/set2-INAP/tester_app/gb001_a_party_hangup.lua

[1/1][START ../test/regression/set2-INAP/tester_app/gb001_a_party_hangup.lua]
[STARTED tr_172115_7098_1 ...]
[COMPLETED 1(1) of 1 instances in 0.090 seconds.  Aborted = 0, Failed = 0, Cancelled = 0, Rate = 11.152/sec]
>>> FIRST CALL CHECKS
... [06] [PASS] Received INAP opcode FurnishChargingInformation [34] as expected.
... [06] [PASS] String 'FCI.[0].tag_hex' is present and has expected value '001c'.
... [06] [PASS] String 'FCI.[0].value_hex' is present and has expected value '0100ffffffffffffffffffffffffffffffffffff'.
... [06] [PASS] String 'FCI.[1].tag_hex' is present and has expected value '002c'.
... [06] [PASS] String 'FCI.[1].value_hex' is present and has expected value '0200000000000000001000000000000000000000'.
... [06] [PASS] String 'FCI.[2].tag_hex' is present and has expected value '004c'.
... [06] [PASS] String 'FCI.[2].value_hex' is present and has expected value '0000000000000017228700000049300000005042'.
... [06] [PASS] String 'FCI.[3].tag_hex' is present and has expected value '999c'.
... [06] [PASS] String 'FCI.[3].value_hex' is present and has expected value '0000000000000000000000000000000000000000'.
... [09] [PASS] Received INAP opcode ApplyCharging [35] as expected.
... [09] [PASS] Integer 'R1.maxConversationTime' is present and has expected value 120.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.warningTimeBeforeExpiry' is present and has expected value 110.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.messageID.elementaryMessageID' is present and has expected value 500.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.numberOfRepetitions' is present and has expected value 1.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.duration' is present and has expected value 0.
... [09] [PASS] Integer 'R1.treatmentForFirstWarning.informationToSend.inbandInfo.interval' is present and has expected value 0.
... [12] [PASS] Received INAP opcode RequestReportBCSMEvent [23] as expected.
... [12] [PASS] Integer 'BCSM(5,2)monitorMode' is present and has expected value 0.
... [12] [PASS] Integer 'BCSM(6,2)monitorMode' is present and has expected value 0.
... [12] [PASS] Element 'dPSpecificCriteria' exists as expected (and is defined).
... [12] [PASS] Integer 'BCSM(6,2)dPSpecificCriteria.applicationTimer' is present and has expected value 300.
... [12] [PASS] Integer 'BCSM(10,1)monitorMode' is present and has expected value 1.
... [12] [PASS] Integer 'BCSM(4)monitorMode' is present and has expected value 0.
... [12] [PASS] Integer 'BCSM(9,2)monitorMode' is present and has expected value 0.
... [12] [PASS] Integer 'BCSM(9,1)monitorMode' is present and has expected value 1.
... [15] [PASS] Received INAP opcode Connect [20] as expected.
... [15] [PASS] String 'destinationRoutingAddress_digits' is present and has expected value '049340221'.
... [15] [PASS] Integer 'destinationRoutingAddress_noa' is present and has expected value 3.
... [16] [PASS] This is a static pass string.
... [17] [PASS] Correctly received DRA digits 049340221.
>>> First Instance all 30 tests returned PASS.

Multiple Runs

Each Lua filename specified on the command line is a single Test Run.

Each Test Run is initiated by sending a StartRun REST request to the RestServerApp.

By default, each Test Run is run in sequence. A new Test Run is started only after the previous Test Run has been reported as completed by a CheckRun REST request. However, the --parallel command-line option will instruct lua_test to run more than one Test Run in parallel.

When executing more than one Test Run, an additional print-out at the end of all test runs will be displayed, giving a summary of the test run results.

./lua_test ../test/regression/set2-INAP/tester_app/*.lua
...

SUMMARY OF 18 RUNS.
  IDX 1st-PASS/FAIL : #START #FINISH #ABORT #FAILED #CANCEL : LUA Filename
    1       30    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb001_a_party_hangup.lua
    2       31    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb002_b_party_hangup.lua
    3       33    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb003_timer_expires.lua
    4       40    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb004_post_call_interaction.lua
    5       28    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/gb100_simple_charged.lua
    6        5    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap001_idp_ext_rc_logic.lua
    7       16    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap002_ctr_pa_pacui.lua
    8       12    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap003_sinap_fci.lua
    9       11    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap004_etc_pa_pacui.lua
   10       18    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap004_etc_pa_pacui_double.lua
   11        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap005_etc_pa_pacui_error.lua
   12        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap006_etc_pa_pacui_abort.lua
   13       14    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap007_connect_bcsm_cir.lua
   14        6    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap100_simple_connect.lua
   15        4    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap101_vars_connect.lua
   16        4    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap102_idp_abort.lua
   17        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap200_intester_camel.lua
   18        9    0        1       1      0       0       0   ../test/regression/set2-INAP/tester_app/inap400_intester_camel4.lua

Common Options

The following common options may be specified when executing lua_test.

Option Type Description
--host String Specify non standard REST RPC server listen host for RestServerApp.
(Default: localhost)
--port String Specify non standard REST RPC server listen port for RestServerApp.
(Default: 9010)
--quiet Switch Suppress reporting of the first call debug.
Suppress reporting of the first call passed tests.
Suppress reporting of intermediate progress lines during the test run.
--minimal Switch Suppress additional reporting during test run execution.
--space Switch Inserts whitespace between test results when executing multiple tests.
--failure-only Switch When executing multiple tests report only failed tests in the final execution summary.
--summary-totals Switch Compute and print the sum of tests and individual check statements as a summary line at the end.
--parallel Number Set the number of test runs to execute simultaneously. Default = 1.
Applicable only when more then one Lua test run script is specified on the command line.
--google-test-output String Request a Google test output formatted XML file to be generated to the provided path.
Google test XML may be used by downstream services such as Teamcity for reporting.
--trace_level Number Requests that tracing be enabled for Test Run Instances executing within the LogicApp for this test run.
--trace Number This is an alias for --trace_level.
--debug Switch This is the same as setting --trace_level 1.
--dump Switch This is the same as setting --trace_level 2.
--spam Switch This is the same as setting --trace_level 3.

Load Testing Options

These options specify that each single Test Run will execute the exact same test script multiple times.

Option Type Description
--total Number Determines the number of total test iterations that lua_test will perform for each Test Run.
If specified as a value greater than one, then at least one of --cps or --backlog must also be specified.
--ramp Number Increments the concurrent number of calls by the specified value of --ramp over the duration of the performance test. This is intended to be used with --total and a combination of either --cps or --backlog, otherwise only one test will be performed.
--cps Number Executes a concurrent number of calls equal to the --cps value. If no --total is provided the total number of calls will be ten times the value of --cps.
--backlog Number Instructs lua_test to process tests as fast as possible, having a minimum of the --backlog specified value in progress at one time. If no --total flag is provided, the total number of calls will be ten times the value of --backlog.
--load-details Switch Allows the matching of inner JSON test operation content under load test conditions.

Variables

The lua_test may pass additional variables into the script execution.

These are passed as the extra_args (second parameter) into the Lua chunk code.

There is a mechanism for providing all of the variables, and additionally there are various short-cut command-line options to set some commonly used variables. These short-cut variables are applied after the base variables Object has been set.

All of these variables have no intrinsic “special” behavior. They all require the executed script to reference them in the appropriate way e.g. when constructing the SCCP Destination Address for sending an InitialDP operation.

Option Type Description
--variables String (JSON Object) A string representing an Object in JSON notation.
--variables-file String (File Path) The path to a file which contains an Object in JSON notation.
--dpc Number Defines a dpc attribute within the scp Object variable.
--dssn Number Defines a dssn attribute within the scp Object variable.
--dgt_digits String Defines a dgt_digits attribute within the scp Object variable.
--dgt_noa Number Defines a dgt_noa attribute within the scp Object variable.
--dgt_np Number Defines a dgt_np attribute within the scp Object variable.
--dgt_tt Number Defines a dgt_tt attribute within the scp Object variable.
--smpp String Defines a smpp attribute within the route Object variable.
--pi String Defines a pi attribute within the route Object variable.
--soap String Defines a soap attribute within the route Object variable.
--diameter String Defines a diameter attribute within the route Object variable.
--sigtran String Defines a sigtran attribute within the route Object variable.
--rest String Defines a rest attribute within the route Object variable.

SCCP Variables

The command-line short-cuts for dpc, dssn, dgt_digits etc. is designed to be compatible with a script similar to the following example:

local rest_details, variables = ...

-- Default SCP Destination Routing Address
local scp_dra_prefs = { dpc = 4114, dssn = 106 }

-- Construct the SCCP DRA from variables.scp (with fallback to defaults).
local scp_dra = tcap.sccp_address_merge (variables.scp, scp_dra_prefs)

-- Initialise the TCAP Transaction using the constructed SCCP DRA
tcap.initialise (nil, 'ssp.scp.1', scp_dra, tcap.APPLICATION_CONTEXT_ALIASES['camel3'])

Overriding the default SIGTRAN route with the --sigtran command-line short-cut variable would assume:

local rest_details, variables = ...

-- Apply a possible SIGTRAN route override variable.
local sigtran_route = nil
if (variables and variables.route and variables.route.sigtran) then
sigtran_route = and variables.route.sigtran
end

-- Initialise the TCAP Transaction using the constructed SCCP DRA with possible SIGTRAN route
tcap.initialise (sigtran_route, 'ssp.scp.1', scp_dra, tcap.APPLICATION_CONTEXT_ALIASES['camel3'])