Cyber defence exercises are instrumental for enhancing capabilities and expertise in the cyber domain. Cyberspace as an operational domain is acknowledged by NATO, alongside land, sea, air and space. As NATO nations are endorsing the development of both defensive and responsive cyber capabilities, it is reflected in growing demand for appropriate training events. It is through training and exercises that the skills, readiness and awareness of experts can be augmented.
The increased training need requires cyber defence exercises that are professionally organized and customized for the specific training audience. It takes skilled experts to develop the scenario and build up the required infrastructure, but also to include elements to the training challenge that motivate training audience to give of their best. The researchers at the NATO CCDCOE have found that adding competitive elements to the cyber exercise is beneficial and serves this purpose well. The competition can be scored based on the performance of training audience by checking how well they keep their systems up and functional.
In this paper, NATO CCDCOE researcher Mauno Pihelgas presents an overview of the availability scoring system for a large international live-fire cyber defence exercise Locked Shields. It discusses the design and practical implementation of the system and highlights some key observations based on the experience from exercises organized by NATO CCDCOE over last five years.
In particular, the paper focuses on different elements of designing and building a reliable availability scoring system for Locked Shields exercise. The developed system provides an essential input for scoring the exercise participants to spark some friendly competition and motivate players. The previous solution was replaced by a modular setup that is built around a well-known open-source IT monitoring software called Nagios Core. Before embarking to develop a new system, the authors studied available research and looked at various other CDXs for similar implementations.
The paper provides some background information on the exercise, describes the requirements, design process and implementation of the scoring solution. The system presented in this paper has been under continuous improvement since 2014 and has been successful in providing the automated scoring checks for the past five iteration of Locked Shields. In addition to success stories, several issues and problem workarounds are addressed. As such, this paper serves as a valuable resource for cyber defence exercise managers and practitioners looking to implement similar scoring solutions.
Keywords: availability, cyber exercise, monitoring, Nagios, scoring, Selenium