ETSI integrates its NGSI-LD conformance test suite with the Interoperability Test Bed

The European Telecommunications Standards Institute (ETSI) has completed a proof of concept connecting its official NGSI-LD conformance test suite with the Interoperability Test Bed. Furthermore, a second phase of the work is starting in September 2026, in view of advancing this integration and producing a full ETSI specification.

The European Telecommunications Standards Institute (ETSI) is a European Standardisation Organisation producing globally applicable standards for Information and Communication Technologies. Among the specifications maintained by ETSI – and specifically the Technical Committee on Data Solutions (TC DATA) – is  NGSI-LD, a specification defining an information model and API for context information management.

NGSI-LD enables the publication, discovery and consumption of the real-time state of entities such as devices, infrastructure or environmental sensors. This data is summarily referred to as context information, whereas services implementing the information exchange API are defined as context brokers. NGSI-LD is accompanied by an official test suite – based on the Robot framework and published on the ETSI Forge – as a reference tool for assessing implementations’ conformance. 

In view of complementing the official test suite, ETSI launched in September 2025 a study and proof of concept to explore a possible integration with the Interoperability Test Bed (ITB). The goal of this was to improve user experience and extend the test suite’s distribution and integration options, without duplicating or rewriting the existing test logic. The question underpinning the study was essentially “how can we benefit from the ITB without redoing existing work?”.

The key constraint of the work was that the Robot framework test suite would remain the single source of truth for NGSI-LD conformance. Replacing the test suite with the ITB’s own Test Description Language (TDL), or maintaining a parallel copy, was never an objective. The approach studied was to generate ITB test cases from the Robot framework tests, using ultimately both sets of tests in parallel with different responsibilities: a Robot framework test case handles test step execution and assertions, whereas an ITB test case triggers the overall test and displays results.

Two candidate patterns were considered. The first one was a coarse-grained approach that maps an entire run of the Robot framework test suite to a single ITB test case. This is inexpensive to set up but suffers reporting-wise as it collapses entire test sessions into single steps, while producing one overall verdict. The second – and ultimately selected – pattern, was a fine-grained approach that maps each Robot framework test case to a separate ITB test case. Although being more complex to implement, this results in a near-native ITB experience, presenting distinct test cases and executed steps with detailed per-step reports.

The proof of concept demonstrated the feasibility of the approach end to end, generating ITB test cases across the full test suite with no duplication of test logic. Moreover, the selected approach required neither ITB-aware extensions to the Robot framework tests, nor adaptations to the ITB’s core. The integration – explained further below – is based on existing TDL reporting steps and the ITB’s standard extension mechanism to drive the tests.

The key element of the integration is a custom connector, implemented as a standard ITB extension service, which runs alongside the ITB and the official test suite. Its job is to bridge the two systems by means of two separate flows.

The initialisation flow comes first, and runs each time the official test suite changes. The connector determines each test case’s structure by means of a listener-based dry-run execution, which generates an ITB test case per Robot framework test. The generated tests are automatically packaged and deployed to the ITB using its REST API, resulting in a process that can be ran on-demand or fully automated and triggered when changes are detected.

The execution flow follows, whereby the ITB is used to launch one or more test cases. Launching a test calls the connector, which in turn triggers the corresponding Robot framework test against the system under test – the context broker being tested – and records each step’s inputs, outputs and outcome. These are then returned to the ITB’s test engine, which updates the displayed test steps with their relevant results. Any configuration values needed by the test are supplied through the ITB's own configuration features rather than hard-coded in the tests.

Executing tests in this manner fully covers both goals: NGSI-LD’s Robot framework test suite remains the single source of truth, while being managed and presented through the ITB with a near-native ITB user experience. Beyond this implementation, the proof of concept also identified future extensions, including extending coverage for complex asynchronous test scenarios and switching to reporting test progress per test step rather than only once the full test completes.

The results of the study and proof of concept are detailed in ETSI Technical Report TR 104 186 which is currently pending final approval and publication. Besides covering the ITB integration, the report also sets out suggested enhancements to the NGSI-LD test suite for future integration, including stable and unique test identifiers, and a consolidated catalogue of configuration parameters.

Building on this outcome, ETSI launched in May 2026 a Call for Expertise for a new Testing Task Force (TTF) on Automating Conformance and Interoperability Tests, with work planned to start in September 2026. This TTF will take forward the extensions to the ITB integration identified during the proof of concept, with the intended outcome of being a full, published ETSI specification, that can be applied to any existing testing implementation. This would make the approach taken for NGSI-LD a generally applicable blueprint that can apply to any specification seeking to extend their current testing setup to benefit from an integration with the ITB.

More information on ETSI’s mission and activities is available on its official website, whereas for the NGSI-LD specification you can refer to its information portal, normative specification, and official test suite. Regarding the Interoperability Test Bed itself, general details on its services and use cases can be found in the Interoperable Europe Portal, with its value proposition being a good starting point for newcomers. To receive updates on Test Bed news and releases remember to subscribe to the Test Bed on the Portal, and to follow Interoperable Europe's updates on X and LinkedIn.

The Interoperability Test Bed is a service provided by the European Commission's DIGIT, offering conformance testing and validation solutions in support of IT systems' cross-border interoperability.