Case Study on Artemis Security Scanner

This case study has been drafted based on comprehensive desk research—including official documentation, technical portals, and academic articles—as well as two interviews conducted by the OSOR Team with Krzysztof Zając, project leader of Artemis at CERT PL, and Adam Kliś, creator of Artemis and president of KN Cyber.

In July 2016, the European Commission adopted the first EU-wide horizontal legislation on cybersecurity: the Directive on Security of Network and Information Systems (NIS). The Directive aimed to “achieve a high common level of security of network and information systems within the EU” by improving cybersecurity capabilities at the national level, increasing EU-level cooperation, and managing risks and incident reporting obligations for operators of essential services and digital service providers [1].

Under Article 7 of the NIS Directive, every EU Member State had to adopt a National Cybersecurity Strategy. Consequently, Poland adopted the National Cybersecurity System Act on July 5, 2018, with the main objective of creating a national cybersecurity system with a precise distribution of tasks and obligations across designated actors [2]. The Act was amended on April 3, 2026 to comply with NIS 2 updates, which revised the NIS Directive to address inconsistencies in national implementation, enforcement failures, and a narrow scope that excluded vital sectors such as public administrations or waste management [3].

Central to the Polish framework was a shift from reactive defence to proactive threat hunting or, in other terms, neutralising vulnerabilities before attackers can exploit them. To execute this mandate, the Act designated three computer security incident response teams (CSIRTs), across three different Polish institutions: (1) the Internal Security Agency (GOV), (2) the Research and Academic Computer Network (NASK), and (3) the Ministry of National Defence (MON).

Article 26 of the Act assigned these CSIRTs different tasks, including national threat monitoring, advanced malware analysis, tool development, and cybersecurity awareness initiatives. Within NASK, vulnerability analyses are conducted by CERT PL through Artemis [4].

Artemis is an open source modular security vulnerability scanner operated by CERT PL, which is Poland’s first Computer Emergency Response Team (CERT) functioning since 1996 as a specialised division within CSIRT NASK. Artemis performs regular and automatic scanning of the network of the public entities that fall under its constituency, including hospitals, schools, research institutions, universities, and local authorities, as well as on-demand scans for companies that submit their domains through the designated portal, to detect misconfigurations, out-of-date software, and security bugs on a large scale [5]. At its core, Artemis serves to protect public and private sector organisations that are used by citizens daily from cyber threats and vulnerability exploitation. 

Artemis is released under the BSD 3-Clause Licence, which is the standard licence under which CERT PL releases its open source software. The licence allows external organisations, corporations, and other national CERTs to study, modify, and run the software both publicly and commercially. To enable this, CERT PL maintains an updated repository of Artemis’ materials, accessible on GitHub.

The security scanning begins with data ingestion, where a targeted list of domains or IP addresses provided by the CERT PL team is inputted into Artemis. CERT PL obtains the domains to scan from two main sources:

1. Publicly available lists of domains of Polish public administration bodies and entities, such as government agencies, hospitals, or schools; and

2. Their self-service portal, moje.cert.pl, which allows private entities and individuals to register their domains and have them scanned by Artemis. Currently, more than 20,000 entities have registered their domain in the self-service portal. 

Once the domains are identified, the system kicks off an automated reconnaissance phase called subdomain enumeration to build a comprehensive map of an organisation’s digital presence. This process performs subdomain discovery and forwards the resulting targets to a network profiling stage, where open server ports are identified, and active software services are detected.

Once the digital infrastructure has been mapped, the scanning engine dispatches the targets to specialised, independent testing modules, each activated by the presence of specific technologies on government servers and executed in parallel in the background. The list of modules includes, among others:

• CMS app scanners

• Content router and exploit scanners

• Path and directory solvers

• Authentication and login auditors

• DNS and domain infrastructure analysers

• Deep injection and advanced testing. 

The complete mapping of modules and their description is available on the Artemis website. 

The automated modules then perform targeted and non-disruptive security checks tailored to the technologies detected on each server. For example, they can identify the specific versions of web content management systems such as WordPress and Joomla or verify whether email servers are properly configured to prevent spoofing. In parallel, deeper probing modules scan for common but high-risk security flaws, including accidentally exposed backup files and configuration folders, Domain Name System (DNS) misconfigurations, structural web application vulnerabilities such as Structured Query Language injection (SQLi), and weak or default credentials on administrative login portals. The box below provides more information on some of these common vulnerabilities detected by Artemis. 

To ensure the system remains practical, all raw data gathered by the modules is passed through a strict post-processing routine. This phase filters out false positives and automatically cross-references and merges overlapping results, ensuring that duplicate findings for the same vulnerability are discarded if they have already been logged or reported.

Example of Artemis scanning results. Source: The Artemis security scanner | CERT Polska

In the final stage of the process, Artemis maps all unique verified security flaws onto a severity scale (high, medium, low) and automatically packages them into clean and actionable summary reports. These reports explain each identified vulnerability and the steps needed to address it in both technical and non-technical terms. 

Summary of the Artemis process. Source: Artemis Security Scanner

Importantly, Artemis has clear safeguards in place to ensure that its scanning activity is not mistaken for malicious behaviour by the organisations it assesses. Scans are conducted from a single IP address, allowing administrators to verify that a given request originates from Artemis. In addition, vulnerability reports are consistently sent from the same email address, cert@cert.pl [6].

Artemis was first conceptualised and developed in November 2020 by the KN Cyber Science Club at the Warsaw University of Technology, where student researchers built the prototype with the goal of scanning their university’s network for vulnerabilities. The students soon realised the many benefits that the tool could bring to the broader Polish public administration and pitched the project to CERT PL, which recognised the value of the solution and took over its development to turn it into production-grade software in August 2021. 

In January 2023, CERT PL decided to move Artemis into deployment despite recognising that it was not yet a fully mature solution. The rationale was that, even in its early form, Artemis could already generate valuable leads on network vulnerabilities. This early deployment enabled an agile learning process and rapid scaling, as the tool could be continuously refined on the basis of real-time feedback and performance assessment.

Since CERT PL assumed responsibility for Artemis’ development and management, the team behind the solution has evolved in line with changing operational needs. In the early stages, the focus was primarily on further developing and refining Artemis by adding new features, implementing modules, strengthening the underlying software engineering, and scaling its infrastructure. At the same time, because the solution was not yet fully established, it required substantial administrative support, particularly for identifying the appropriate contact points within scanned organisations and managing incoming enquiries. As Artemis matured and became known, and the CERT PL team developed a contact directory, the nature of the work gradually shifted towards operations, including maintenance, addressing performance regressions, and managing the self-service portal, which has helped expand the tool’s coverage.

In terms of financing, Artemis was initially created with a very limited budget of 30 euros fronted by KN Cyber and used to purchase a Virtual Private Server (VPS). The handover to CERT PL in August 2021 did not involve any monetary transaction between parties. There is no concrete figure regarding the cost of the solution, but interviewees confirm that “it is not an expensive project to run”. Currently, Artemis runs on an operational budget that mostly comes from CERT PL’s budget for NIS and NIS2 implementation, as Artemis matches well the requirements set out by these directives. 

One of Artemis’ most notable achievements is the number of vulnerabilities it has identified and reported within only a few years of operation. As of July 28, 2026, Artemis has reported 1,736,000 vulnerabilities, of which around 125,000 are of high severity. These vulnerabilities were identified not only in public administration networks, but also in high-profile private-sector organisations, including national banks and newspapers. This demonstrates Artemis’ relevance for strengthening the security of both government and commercial networks. More broadly, one of Artemis’ key achievements has been helping organisations recognise that their networks are often more exposed than they assume, and that administrators may not be aware of many of the vulnerabilities present in their systems.

As highlighted by both interviewees, one of Artemis’ most significant benefits is the quality of the reports it produces. Generated automatically in either Polish or English, these reports explain identified vulnerabilities in both technical and non-technical terms, enabling administrators to understand the issue and take the necessary steps to address it. In most cases, the reports focus on “low-hanging fruit”: misconfigurations and minor errors that may appear small in isolation but can still compromise important networks if left unresolved. 

Another significant benefit of Artemis is its modularity and scalability. Its open source technical architecture and governance model enable emergency response teams other than CERT PL to extend and improve the solution by developing modules tailored to their specific needs. When these modules are released back to the community, they help broaden Artemis’ functional scope and enhance its overall performance. For instance, community-driven integrations like sqlmap (SQLi discovery), wpscan (WordPress vulnerability scanning), or ssl_checks (Secure Sockets Layer/Transport Layer Security misconfiguration testing) are shared back via the official Artemis-Modules-Extra repository, allowing any team to instantly deploy these third-party tools. In addition, Artemis includes highly configurable rate limits by default, which allows custom scanning speed and depth to match the capabilities of scanned organisations. This enables entities with limited infrastructure to be scanned by CERT PL or run Artemis themselves without overwhelming their systems.

In terms of challenges, the manual identification of the administrators responsible for addressing the vulnerabilities detected through its scans remains an operational bottleneck. However, the issue is being progressively mitigated as CERT PL collects and maintains contact information for the entities it scans in an updated directory. In addition, as Artemis scales and its user base expands, CERT PL is increasingly facing the challenge of maintaining scanning speed and performance. Because CERT PL runs Artemis on seven standard, in-house machines, hardware limitations impose a natural ceiling on both scanning speed and overall capacity. 

Artemis has already achieved comprehensive coverage of Polish public entities, and therefore CERT PL is not proactively expanding the list of domains it scans. Rather, CERT PL is seeking to boost registration in their self-service portal, moje.cert.pl, through various outreach activities. 

Research is also underway to complement Artemis with agentic vulnerability detection tools for specific scenarios. CERT PL recognises that AI tools cannot replace Artemis as a core vulnerability scanner, as the high computational costs and resource demands of large language models make them impractical for mass-scale, continuous scanning. However, rather than viewing AI and deterministic scanners as mutually exclusive, CERT PL treats them as complementary technologies: while Artemis efficiently handles high-volume scanning, AI agents can step in for nuanced tasks such as contextualising complex findings, prioritising alerts, or filtering false positives. CERT PL is exploring these complementarities to enhance their capabilities without compromising performance and substantially increasing costs. 

• Organisations wishing to have their networks scanned by CERT PL can register their domains at moje.cert.pl.

• Developers or institutions interested in deploying, modifying, or extending Artemis can access its open source repository on GitHub.

For information regarding the policy context of open source software in the Polish public sector, please consult the respective Country Intelligence Report and its corresponding factsheet in the OSOR Knowledge Centre. These resources include a detailed overview of the political actors, strategic players, political and legislative frameworks and public sector open source software initiatives in Poland. 

Click number to go back to text

[1] European Commission. (2018, May 4). Directive on Security of Network and Information Systems, the first EU-wide legislation on cybersecurity (MEMO/18/3651). European Commission Press Corner. https://ec.europa.eu/commission/presscorner/detail/en/memo_18_3651 

[2] Rutecka, D. (2019, January 18). The Polish Act on Cybersecurity: Initial remarks. Schönherr. https://www.schoenherr.eu/content/the-polish-act-on-cybersecurity-initial-remarks 

[3] Dispel. (2024, July 29). What was the original NIS Directive, and why was it not sufficient? https://dispel.com/blog/what-was-the-original-nis-directive-and-why-was-it-not-sufficient 

[4] CERT Polska. (2023, January 25). Artemis – CERT Polska verifies the cybersecurity of Polish organizations. NASK National Research Institute. https://cert.pl/en/posts/2023/01/artemis-scanning/ https://cert.pl/en/posts/2023/01/artemis-scanning/

[5] CERT Polska. (2023, January 25). Artemis – CERT Polska verifies the cybersecurity of Polish organizations. NASK National Research Institute. https://cert.pl/en/posts/2023/01/artemis-scanning/ 

[6] CERT Polska. (2023, January 25). Artemis – CERT Polska verifies the cybersecurity of Polish organizations. NASK National Research Institute. https://cert.pl/en/posts/2023/01/artemis-scanning/ https://cert.pl/en/posts/2023/01/artemis-scanning/