CPSV-AP Harvester

[This tool is archived. This means that the tool is no longer actively maintained by SEMIC, but still available for use at own risk.]

Table of Content

1. Introduction
2. Technical Documentation
3. Deployment
4. Adaptability

Introduction

The CPSV-AP Harvester allows public administrations to obtain the data faster and more efficiently, easing creation of an integrated view on life events, business events and public services provided within a specific country or region. Using the Harvester, public administrations can automatically update public service descriptions owned by different competent authorities on multiple levels of administration (national, regional, local) within a central catalogue.

The test implementation of the CPSV-AP Harvester allows the user to harvest machine-readable files described in RDF/XML, RDF Turtle or N-triples while storing the data in its own triple store, giving the user the possibility to browse through all the harvested data in a single place.

The Harvester webpage acts as the user interface; this needs to be configured by an administrator, pointing the CPSV-AP Harvester to the correct database or URI to harvest the public service descriptions.

Technical Documentation

This section provides the technical details of the CPSV-AP Harvester Tool.

The Harvester works as a script. Its main functionality is processing the different sources indicated in a configuration file, gathering the available data and storing it in a central triple store. Note that the data source needs to be configured to permit the Harvester access.

The figure below provides a simplified schema on how the Harvester tool works. 

As can be seen, the technical architecture consists of:

• A configuration file, to indicate the configuration data;
• A website for the user interface, to launch the harvesting and visualise the data, and
• A Virtuoso triple store, to provide a database to store the information harvested.

More detailed information on the technical specification of the CPSV-AP Harvester can be found in the document available here.

​Deployment

Web server

The quickest and easiest way to use the tool is by accessing the deployed test implementation of the CPSV-AP Harvester on the Amazon Web Server. This can be accessed here.

Please note that the web server is for testing purposes only.

Local deployment

The code of the test implementation of the mapping tool is available on GitHub here.

For detailed steps for deployment, please refer to the guide, which can be downloaded here.

​Adaptability

The test implementation of the CPSV-AP Harvester can be easily adapted by updating the JavaScript file and/or the HTML webpage.

To add new functionalities, the programmer has to update the JavaScript file to include new actions, i.e. the “cpsv-ap_harvester.js” file. This file is located under the root folder. In addition, they may have to modify the HTML page, if required.

When implementing new functionalities related to the triple store, the user will also have to create the corresponding PHP and python files to execute the queries needed, place them under the “pages” folder and invoke the PHP from the JavaScript.