Peer learning session #4 “Data catalogue enablers” event highlights

We are committed to align our solutions with the European Interoperability Framework. Accordingly, I believe the reusability of the solution can be assessed both from a technical standpoint, such as through the platforms we have developed and released as open source, and at a systemic level, in terms of platform functionality and the technical solutions implemented. This approach facilitates a more straightforward assessment process, as our adherence to the European framework and industry standards minimises unfamiliar elements. Should there be any questions or concerns, we remain available for further discussion.

The primary challenge was not technical in nature. Established solutions were already in place, so developers encountered routine tasks within their daily responsibilities. The key issue was persuading administrative stakeholders to trust and adopt the platform. Despite legal requirements mandating its use, adoption was contingent on administrators' confidence in the platform’s reliability. Therefore, demonstrating the platform’s robustness and encouraging a shift to new practices were crucial steps. As administrators began to recognise the platform’s value, they started requesting its integration into new projects, validating its utility and strength as a viable solution.

Additionally, implementation revealed further considerations. Engaging with administrative bodies often surfaced regulations or questions that required careful attention. Thus, maintaining open communication and thoroughly explaining how the solutions function proved essential, as understanding may not be immediate for all parties involved. This process required significant time, more than what was spent developing the platform itself.

We are actively seeking use cases to evaluate our cross-border solution. Our architecture is built on robust assumptions and established standards, and we would like to test its effectiveness through sandbox environments as well as real-world scenarios. As previously mentioned, it is essential to establish a framework that ensures legal data sharing, supported by specific use cases for data exchange. We are currently engaging with several administrations, but our primary objective is to enable a public, central administration, such as one in Germany, to securely access our National Registry for purposes like verifying residency or family status. This process involves several steps, yet it holds substantial promise. Additionally, we aim to assess the solution both independently and in conjunction with connectors designed for integration with existing systems.

Currently, the platform operates in a centralised manner. However, it does not entail central governance. There is no authority determining which data is shared with specific parties. Instead, each participant utilises the platform independently, making decisions regarding data accessibility and usage. The platform functions as an enabler, with governance limited to adherence to platform rules. The new solution under consideration is designed to be scalable, accommodating even a relatively small number of entities, and aims to facilitate operations across various countries and member states. This approach supports replicability, allowing for multiple platforms within a single member state if necessary. At present, we maintain a centralised platform without centralised governance.

The current cross-border data exchange is in active development, with the primary challenge being the authentication of parties and assignment of attributes. It is essential that any solution provides robust mechanisms for party authentication, which can be achieved through a variety of means. The new implementation focuses on facilitating certain types of interactions and remains open source, allowing for flexible deployment without hidden methods or mandatory protocols. Users may choose our solution or utilise the token exchanged between systems. The process aligns with standardised practices and preserves sovereignty by avoiding enforcement of a fixed approach.

For instance, two distinct gateways were created as open-source projects to enable interaction with the platform, without requiring adherence to a singular solution for local configurations. These gateways, widely adopted within our administrations, serve as local proxies or interfaces for platform communication. The successful development and use of these two open-source solutions demonstrate the platform's openness and reliance on implementable standards.

The platform is open source and publicly available, allowing for thorough evaluation of its technical reusability as well as its overall system architecture. It implements standardised interaction types, supporting digital sovereignty by avoiding proprietary or restrictive methods. Member states or administrations with limited interoperability experience can leverage our solution and its open-source connectors to establish a comprehensive system without the need for entirely new installations. Our team remains available to provide further information and support as needed.

The onboarding process was conducted in accordance with applicable legislations. When the decision was made to develop the platform, a law stipulated that administrations must publish their data via the Digital National Data Platform. Although they may continue to share their data through other channels, compliance with the Digital National Data Platform is required, and a deadline was established. The implementation remained challenging, despite this legal mandate, due to the existence of prior laws advocated by some stakeholders.

This approach proved to be highly effective for onboarding, particularly within central administrations. For other administrative bodies, we allocated an excess of five hundred thousand euros from NextGenerationEU funds to support their acquisition of solutions, integration with the platform, and overall usage. It is important to note that smaller administrations tend to act primarily as data consumers rather than data providers, despite the platform's federated and collaborative framework. These smaller entities leverage the platform to facilitate interoperability among their various software solutions.

Regarding procurement, our approach was consistent with established procedures. The procurement guidelines specified the use of the national platform as an initial step. Subsequently, we allocated funds to support central administrations in transitioning to the platform, as well as to collaborate with them in planning, training, and ensuring proper implementation. We continuously identified areas for improvement and enhanced the platform accordingly. This process involved ongoing collaboration and partnership with various administrations to facilitate their adoption and effective use of the platform.

Access policies are determined by individual entities and service providers, which is one of the reasons behind the project's success; administrations value having ownership over their data. Understandably, they are protective of their information. Therefore, our role often involves facilitating communication between consumers and providers to encourage data sharing. Policies are structured around user groups and attributes, with these attributes defining group membership, regardless of whether the entity is a service provider or a private organisation. Entities can select which groups may access their data, though some administrations preferred granting permissions on an individual basis, a process that proved to be time-consuming.

Once users began to develop greater trust in the platform, and as the volume of approval tasks increased, they recognised the potential for implementing automatic approvals. This could be achieved by forming more focused groups with automated approval mechanisms, thereby streamlining administrative processes. Now that the platform is established, with all data and APIs available, current work often involves cases where specific administrations request access to certain datasets, occasionally requiring facilitation between parties. As previously mentioned, each administration retains ownership of its respective data, which is an important principle, though establishing this framework may require time.

It is necessary to determine whether there is an applicable law, whether the law can be interpreted differently, or whether amendments are required. Accordingly, discussions with government authorities may be needed to request specific changes in regulations that would facilitate data exchange. This is our current focus, as the platform exists and significant data sharing is already underway. Another challenge involves the types of APIs available, since stakeholders occasionally require data in formats that are not presently supported. Addressing these concerns is our primary objective at the moment.

Yes, that would be quite easy because our solution is cloud-based. Moreover, user access can be given to our test environment for anyone wanting to test our tools. The source code is available on GitHub, but because it’s a cloud-based solution some parts are not there. However, everything is done with public open-source tools, and it uses the European Public Licence, so information is also accessible there.

Yes, we would be happy to collaborate with other member states or Union entities. For example, there are member states that are using the platform already. Additionally, as our system is modular by design, it is by principle easy to introduce new features to it. For example, now we have a user interface in three languages, because Finland has two official languages, Swedish and Finnish, and then we have English for international use. If someone wants to have another interface, it’s quite feasible to do that and it wouldn’t be too complicated to add a language. Of course that is not done with Finnish funding. If someone wants to use the Interoperability Platform and wants to have new features, then it is more like a problem about the division, of course. In principle we are open to this type of collaboration and primarily need to understand how the Commission can support such multi-country projects. Generally, it depends on what kind of needs come along.

Yes, it’s suitable. It has been used at local, regional and national level, as well as in international and multi-country projects. A central governance is not necessary. It of course needs someone who takes care of the technical maintenance of the platform, like we do, but as mentioned, the maintenance of the contents is up to each organisation. Also, even if a user doesn’t use all the features, partly using the solution still offers benefits, so they don’t have to do everything themselves. For example, in Finland there are some organisations who have only published their terminology on our platform, and that enables them to save costs, since they don’t have to publish those terminologies on their own web pages. 

Most importantly it’s Java-based. It uses microservices, Docker, Amazon Web Services, the cloud, JavaScript, and currently we are using React. 

Our approach is to reuse the European Core Vocabularies. In the past, this was challenging because they were open but not fully resolvable, which made direct referencing difficult.

This has improved with the introduction of the “m8g” namespace, which allows us to reliably reference the European Core Vocabularies. In practice, we first create a local copy of the relevant core vocabularies so they can be resolved and reused consistently. Once they are available locally, we can include them directly in our data models.

For example, we have integrated the European Learning Model (ELM) ontology into our data models. Our data‑modelling tool then exposes the ELM classes, attributes, and associations as if they were native elements of our own system. 

The use of ELM and “m8g” as source for our data models is better illustrated in this core vocabulary example and this application profile example created in our test environment for demonstration purposes.