WienKI: The Holistic Approach of the City of Vienna to Generative AI

Public administrations face the challenge of keeping pace with rapid developments in the field of artificial intelligence (AI) and at the same time using this technology for the benefit of its citizens. The development and commercialisation of powerful generative AI systems resulted in a technological revolution that affects all areas of society, including public administration.

The City of Vienna recognised the need to proactively address this development by working on fundamental questions linked to the responsible and trustworthy use of such emerging technology. How can AI be used responsibly in public administration? How can employees be empowered to use this technology effectively?

Building on these topics and fundamentals, the City of Vienna developed a holistic approach to integrating AI into administrative work. The focus of the project, led by the IT department, was on viewing AI not as an isolated technical solution, but as a tool for modernising and increasing the efficiency of the entire public administration. The challenge was not merely technical but organisational: how to democratise AI access across 50+ municipal departments serving over 70,000 potential users and at the same time maintaining strict compliance with European data protection and AI regulations. 

To address these challenges, the City of Vienna developed WienKI, a holistic approach to integrating AI into administrative work. WienKI is the City of Vienna's internal AI platform, providing user-friendly and customised AI tools for administration, including AI-supported knowledge databases and generative search of various internal data sources accessible through text-to-speech capabilities in 103 languages. The following functionalities are already in production and are used by employees in their daily work:

• AI Assistant: Employees can interact with various language models (GPT, Mistral, Gemini, Llama, and others) to generate formulation suggestions, summarise texts, and much more. Predefined functions allow users without knowledge of prompt-design expertise to benefit from language models. Examples include rewriting passages in plain language, gender-neutral formulation of texts, or checking for word repetitions. The underlying prompts used are visible at any time to ensure complete transparency.

• Knowledge databases: Employees can curate their own knowledge databases by manually maintaining collections of documents. These are made available to the language models as additional context through Retrieval-Augmented Generation (RAG), allowing users to "converse" with their documents. Knowledge databases can be shared as "Smart Info" with other users, and simultaneous interaction with multiple databases is possible. When uploading documents, there is the option to anonymise them on demand so that automatically remove any personal data.

• Image generation: WienKI offers the ability to generate images using DALL-E 3 and GPT Image 1. Images can be saved in a gallery along with the prompt and reused.

In addition, two pilots are currently ongoing: 

• Generative Search (GENSU): Various data sources from the City of Vienna (intranet, wien.gv.at, etc.) will be automatically indexed periodically and can be queried by employees through a search interface. This enables both specialised searches across internal data sources and an enterprise-wide search that can query multiple data sources simultaneously. Additionally, AI provides a readable summary with source references in addition to the search results, significantly enhancing both the accuracy and efficiency.

• Email classification: Municipal mailboxes are automatically read and categorised based on their content and the content of their attachments. This greatly reduce organisational effort, as emails are automatically routed to the responsible employees.

WienKI utilises a hybrid enterprise Azure cloud architecture, hosted on Azure Red Hat OpenShift, with an on-premises SQL (Structured Query Language) Server database for enhanced compliance. Models, knowledge databases, text recognition, speech recognition, and anonymisation are provided using both Red Hat OpenShift AI and Azure services. In addition, databases and Azure Data Lake storage are encrypted to ensure compliance standards.

These technologies enable WienKI to provide modular features whilst ensuring compliance with the EU AI Act, EU Data Act, and GDPR. Through standardised user management via dedicated Microsoft Entra ID tenant and the City of Vienna Data Warehouse (Wien1 User Account), changes can be rolled out in just a few minutes.

The project rests on two pillars, showing that AI adoption isn't just about the tool, but it is about the organisational and regulatory framework that makes safe adoption possible:

1. Strategic control through creation of a legal framework.  Vienna created a legal framework, realising the city’s “AI-compass”, a guideline for the safe adoption of Generative AI (this document can be consulted in the GenAI4PA section of the Public Sector Tech Watch). The city also established an AI Competence Network to tackle AI-related administrative issues from multiple perspectives.

2. Digital transformation through knowledge enhancement and support.  The administration knew it needed to build capacity in AI, so it invested in education and exchange. This includes training at different levels (beginner and advanced), "PrompTogether" sessions where staff learn effective prompting through hands-on experience, and "KI Technologien und Trends" (AI Technologies and Trends) events (KITT). This active AI community has grown within the administration and has become a cornerstone of knowledge sharing, with 1,600 members and an average of 100 participants attending sessions four times per month.

Through the generic modular construction method, WienKI promotes and expands internal systems such as interfaces and data warehouses, increasing interoperability with internal systems and external ones. The platform is indeed designed so that external knowledge sources will also be supported in the future, and Vienna AI can potentially be utilised by other systems. Additionally, the integrated standardised user management ensures that WienKI operates seamlessly within the existing digital ecosystem of the City of Vienna.

In this setting, scalability is by-design: WienKI’s modular architecture allows new open-source content to be integrated at any time. The open-source frontend architecture enables frontend components to be provided as modules. WienKI uses open-source resources from Hugging Face, including Mistral, Llama, Google Gemma, and more. 

The platform is already reused by other administrations within Vienna's municipal structure, including the Vienna Hospital Association, all 50+ municipal departments, and Vienna's social housing organisation.

WienKI is internally accessible to employees, with sources transparently cited, enabling users to jump directly to the relevant document for a more detailed reading. Additionally, prompts and interim results from AI queries are made visible to ensure maximum transparency within the organisations. The digital knowledge-sharing format KITT, with the support of a broad participant network and an active community, adds a significant contribution to the transparent dissemination of AI-related knowledge and best practices in dealing with WienKI within the administration.

The implementation of WienKI has significantly transformed how the City of Vienna's administration approaches AI and knowledge management: 

• Knowledge Democratisation: Over 2,000 knowledge databases containing more than 14,000 documents enable employees to access and interact with organisational knowledge naturally.

• Data Sovereignty and Compliance: By hosting the platform within Vienna's cloud infrastructure and maintaining strict adherence to GDPR, the EU AI Act, the EU Data Act, and NIS2 requirements, the city ensures that sensitive administrative data remains under municipal control.

• Community-Driven Innovation: The KITT community of 1,600 people, with an average of 100 participants attending four sessions per month, fosters knowledge sharing and best practices, creating a culture of continuous learning and innovation.

Drawing from the lessons learned during the development and deployment of the use case, the following challenges were identified by the project team:

• Strengthening the scalability of the AI platform, which is currently bound to the City of Vienna's administration.

• Enabling more employees to use AI and LLMs through organisational change management. Sustaining adoption as the solution scales to additional departments and use cases requires continuous training, support, and cultural change. Not all employees will adopt AI tools at the same pace, requiring differentiated support strategies.

Building on the successful implementation of the project, the following future plans were identified by the project team:

• Developing a new "WienKI 2.1", to include features aimed at improving the platform and adding novel functionalities.

• Moving from a "general chat interface" to the implementation of specific use cases for departments, towards so-called agentic AI.

Useful links:

• Vienna AI website: https://digitales.wien.gv.at/stadt-wien-gewinnt-red-hat-innovation-award-2025/ 

Project contact: 

• Michel Seehuber (IT Department): michel.seehuber@wien.gv.at