Check out the recording and supporting slides for the webinar Immersive realities and location for better public services.

If you are interested in knowing more about ELISE Webinars you can find further information here.

Structure of the presentation

Recording

Summary of the webinar

Over the past years, new visualisation techniques have seen the light: Virtual Reality (VR), which has been used for many years, e.g. in flight simulators, to simulate the approach of an aeroplane to the runaway of an airport. More recently, especially with the development of powerful smartphones and other mobile devices, Augmented Reality (AR) emerged as a technique to combine pictures or videos on the screens with information from the internet (e.g. restaurant, hotel and other information in a street view). 

These techniques are combined and applied in different sectors such as spatial planning, military exercises, aeronautics, environmental monitoring, (city) tourism, management of cultural heritage sites, underground assets, etc. 

There is a huge potential to integrate those techniques in location-enabled public services. The webinar provides examples, zooms in on a few of them, and looks into the key interoperability challenges and ongoing efforts.

The webinar started with a few key take-ways messages: 

• Immersive visualisations exist for some time: first VR, then AR and more and more MR. They very often have a location component. 
• VR, AR, MR … are based on technological developments, both hardware- and data-related developments. 
• Although relatively few are operational, there are many applications, including in the public sector, while more opportunities lie ahead. Examples can be found in security, health, tourism, transport, spatial planning, emergency, education … 

 

Section one (Virtual, Augmented, Mixed and Extended Reality. What are they?) made clear the different forms of immersive visualisations. 

Virtual Reality (VR) refers to a computer-generated simulation in which a person can interact within an artificial three-dimensional environment using electronic devices, such as special goggles with a screen or gloves fitted with sensors. In this simulated artificial environment, the user can have a realistic feeling experience (Mitchell, 2020). Many VR examples can be given from the entertainment sector, the military, aeronautics, commercial applications.

Augmented Reality (AR), on the other hand, is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities including visual, audio … (Schueffel, 2017). The same fields of application emerge here, although tourism and cultural heritage sectors are prominent.

Virtual Reality is a little bit older, and more (scientific) publications exist. According to the Gartner hype cycle, they are both over the top of the curve. Mixed Reality is a blend of physical and digital worlds, unlocking the links between human, computer, and environment interaction. This new reality is based on advancements in computer vision, graphical processing power, display technology, and input systems (Microsoft, 2020). The dimension that is added here is the interaction that takes place between computers and the (physical) environment: it can be location, object recognition, light, sound, etc. Consequently, Mixed Reality is moving between the virtual and real worlds. Extended Reality is used as an umbrella term for all the different types of realities.

Influential in all these developments is that they are based on hardware and data developments. Hardware includes, among others, more powerful computers and computer graphics, gaming consoles, mobile developments (smartphones, tablets, smartwatches…), sensors, holographic & immersive devices, eyewear… and more. 'Soft' developments include modelling, simulation, prediction; gaming industry; 3D visualisation and BIM; Digital Twins; integrated applications; dynamic data flow & big data, etc.

Section two zoomed in on immersive visualisation for public services and highlighted that immersive visualisations are not only limited to the entertainment sector. Several examples were given to illustrate the applicability of immersive visualisation techniques in spatial planning, defence, management of cultural heritage, tourism, management of underground assets and conducting public works, etc.

• The example of the city of Helsinki providing potential tourists with a tour of the city during different seasons was showcased. The Finish VR application starts from the Digital Twin that is being developed for the city. ”Virtual Helsinki - Sound of Seasons” takes the audience through all four seasons of the year. The experience starts with Helsinki’s famous landmark, the empire-style Senate Square and the White Church. The tour continues to the home of architect Alvar Aalto, the father of Nordic design. Finally, you will end up admiring the city skyline and the nature of Lonna Island[2]. 
• The other example provided was in the healthcare sector, a joint initiative of the New York Times, scientists and public sector bodies to create awareness among the public about social distancing and wearing masks and the effect on the spreading (and containment) of the virus. Scientists are learning about the novel coronavirus in real-time, and those who study similar respiratory illnesses say that until it is better understood, no guideline is likely to offer perfect safety. Instead, understanding the possible transmission routes for the virus can help us see why keeping our distance is important[3].
• The third example was on the transport sector and security at airports, particularly in the US; AR is used to screen passengers[4]. 
  

The section ended with the question of how one can measure governments' readiness to embrace those technologies by answering five questions:

1. Who in the organisation requires real-time information?;
2. What are the technology and data requirements? Do we have these?;
3. What are the human resources required?;
4. What are the risks for my organisation in using AR, VR … and
5. What impact will the technology have on my mission-critical activities.

During section three of the webinar, two examples were presented in more detail: one on underground asset management and the other on cultural heritage. 

The first case study is linked to the ARinfuse (Erasmus+ project), which aimed to infuse augmented reality for geospatial information management in the context of utility underground infrastructures. ICT connected with GNSS, GIS and geodatabases, and AR/VR offers the possibility to convert the geospatia

l information of the underground utilities into a powerful tool for field workers, engineers and managers. The project worked on several real-world cases in Cyprus, Belgium and Italy. A series of training modules and an AR tool were developed. Data and technological challenges were also described during the webinar, including how they were tackled in the different pilots. A video was shown on AR for managing the underground in a similar case in Spain.

 

The second case study related to VR application for preserving the cave of Altamira developed and presented by Vincente Bayarri from GIM Geomatics (Spain). The Strategy of the Management System: "Altamira 8D" was presented and discussed and then illustrated through an animation. The data collection and integration cycle was explained and illustrated, resulting in an impressive 3D model of the cave. The whole system is in use by the museum of Altamira with some relevant figures: the premiere in the temporal exposition "The art of reproducing Art" (December 2018) was seen by 100.000 visitors, while the internet videos attracted 74.500 visualisations. 

In section four, some interoperability efforts and challenges were highlighted. The most relevant standardisation initiatives in VR, AR and MR, were presented in the first place. These include SDO's such as ISO - Computer graphics, image processing and environmental data representation (ISO/IEC JTC 1/SC 24), Institute of Electrical and Electronics Engineers (IEEE) with the P2048.1-5 series of standards, as well as ETSI. 

The Open Geospatial Consortium covers several relevant aspects too; its TechTrends watch covers and follows different aspects such as Immersive Visualisation: VR, MR, AR; LIDAR Scanning; Edge Computing; 5G Cellular; Indoor Positioning; 3D Model Creation; Indoor models; Underground; Simulation and Gaming; and Web of Data, Linked Data. Several Standardisation Working Groups are active in these related fields, including the Geopose and MUDDI SWG. Other consortia's role was mentioned: Web3D, KHRONOS Group and the Open AR Cloud Association. One of the major challenges was highlighted: i.e. the discoverability of AR resources.

Key messages and conclusions

The webinar ended with some key take-ways messages and the main challenges that lie ahead.

• Immersive visualisation techniques have existed for some time. They can take different forms: from entirely virtual (VR) to forms that are closer to the real world (AR). Mixed Realities (MR) seem to be the most promising development. 
• VR-AR-MR can be exploited fully when dedicated hardware devices such as specialised eyewear are used. However, also more common devices such as smartphones bring immersive visualisation technology usable by all. 
• Public Authorities and Governments can use the technologies to enhance existing public services or create new innovative services for citizens in many domains. 

The challenges and priorities mentioned were also threefold. 

• VR, AR, MR require high-quality base data, of which a lot is location-based. 3D is a must, while the right attributes must be available to create 'real-world' user experiences. 
• Implementing immersive visualisation technology requires good multidisciplinary collaboration and expertise in the field of ICT, computer vision (VR, AR, MR), location data and technologies, etc. 
• There are still existing barriers that impede the use of VR, AR, MR based content, among others: the discoverability of such content. There is a clear need for standardisation as well, with several SDO's involved.

Although immersive visualisation techniques have existed for some time, they have recently received more attention in the geospatial field (even though the Gartner curve indicates they are in the downwards part of the curve). Immersive visualisation techniques can take different forms: from entirely virtual (VR) to forms that are closer to the real world (AR). Mixed Realities (MR) seem to be the most promising development. VR-AR-MR can be exploited fully when dedicated hardware devices such as specialised eyewear are used. However, also more common devices such as smartphones bring immersive visualisation technology usable by all. Public Authorities and Governments can use the technologies to enhance existing public services or create new innovative services for citizens in many domains such as security, health, tourism, transport, spatial planning, emergency, and education.

The examples shown in the webinar illustrate that VR, AR, MR require high-quality base data, of which a lot is location-based. It is clear, also from the Q&A session, that 3D is a must, while the right attributes must be available to create 'real-world' user experiences. The Altamira case (VR) example shows that implementing immersive visualisation technology requires good multidisciplinary collaboration and expertise in ICT, computer vision (VR, AR, MR), location data and technologies, etc. Finally, there are still existing barriers that impede the use of VR, AR, MR based content, among others: the discoverability of such content. There is a clear need for standardisation as well, with several SDO's involved. From the discussion, it also became clear that public authorities might need some support in the form of guidelines on assessing the readiness of their data (infrastructure) for implementing VR/AR/MR.

From the polls, it can be learned that most attendees work for national public authorities (53%) and that the majority (78%) heard about VR/AR/MR but had a very superficial knowledge before the webinar. The question in which sector VR/AR/MR might be most useful/applicable, spatial planning (35%) and tourism/cultural heritage (25%), scored highest. The most significant barrier to implementing immersive visualization techniques is a lack of expertise and skills (33%). Finally, almost 70% of the attendees said their organisation plans or already tested VR/AR/MR.

 

[1] Doolin, C., Holden, A. and Zinsou, V. (2013). Augmented Government. Transforming Government Services through Augmented Reality (Deloitte). https://www2.deloitte.com/content/dam/Deloitte/us/Documents/public-sector/us-fed-augmented-government.pdf

[2] http://www.virtualhelsinki.fi/

[3] https://www.nytimes.com/interactive/2020/04/14/science/coronavirus-transmission-cough-6-feet-ar-ul.html

[4] https://www2.deloitte.com/content/dam/Deloitte/us/Documents/public-sector/us-fed-augmented-government.pdf