Dr. Matthias Sommer, Dr. Michael Rüegsegger, Operations Research and System Analysis (ORSA), armasuisse Science and Technology

Make better decisions with artificial intelligence

War gaming is an indispensable and versatile instrument for the Swiss Armed Forces, which combines several advantages. Using war gaming, a conflict situation can be simulated and explored. Armed Forces Planning can thus benefit from this method to develop new concepts which it can test quickly in a tactical environment and without much effort. Here, senior staff appreciate the option of reviewing decisions and weighing up the consequences of various different options for action before issuing the corresponding commands. War gaming can thus help, for example, in combating a pandemic to find out which measures need to be taken to provide optimal support to the emergency services in Switzerland, at which time and which location.

Like every method however, war gaming also has certain restrictions. War games are thus driven by the decisions of the players. Each game is unique and cannot be reproduced. In addition, the number of games that are played by human players – in contrast to virtual players – is very limited. This makes it impossible to formulate statistical statements on the outcome of a particular game. Last but not least, the outcome of a war game, as for example in a game of chess, is also heavily dependent on the ability of the players.

Artificial Intelligence (AI) now has the potential to overcome these restrictions and enables war games to be statistically analysed. Using artificial intelligence, digitalised war games can be played and analysed completely automatically, for example. Intelligence agents compete against each other and learn, during innumerable games, which tactical procedures in connection with which military capabilities and technical systems promise the greatest success.

In future, artificial intelligence could thus help persons with decision-making and planning functions to develop new tactics and procedures. For example, to find out which combinations of military abilities are profitable and which requirements are critical for new military systems to be procured.

First application in the area of air defence

To be able to better estimate the potential of artificial intelligence for support in war gaming, employees at armasuisse Science and Technology (S+T) have carried out initial experiments as part of a research project. For this purpose, the war game New Techno War was digitalised and initial algorithms tested. The results were promising and confirmed initial estimations on the optimal playing tactics. However, this game is very abstract and therefore cannot be directly applied to realistic scenarios.

In the next step, it is now aimed to analyse the knowledge gained in the area of air defence more precisely. Due to current procurement projects, the focus was placed on air defence. Three objectives are in the foreground here: First of all, the solution space can be scanned as part of the implementation of “Concept Development and Experimentation” to discover promising new tactics and strategies. This is particularly interesting in the development of operational guidelines for the ground-based air defence systems to be procured. Secondly, specific deployment scenarios in terms of a course of action analysis can be examined and optimised in detail, a core element of the area Operations Research. And thirdly, these methods are also be used in training: By pitting trainees against artificial intelligence, (red teaming), they can train diverse scenarios and thus improve the efficiency of training.

In order to achieve these goals, a computer-based, in other words, virtual model of the battlefield environment to be simulated is required. In addition, an algorithm is likewise required that can learn independently to take successful decisions in this model.

The gaming industry has produced numerous models which are very realistic while at the same time making efficient use of the available computing capacities. This makes computer games interesting candidates for simulating battle scenarios. The computer game “Command: Modern Operations” is suitable for simulating air and naval battle scenarios at a tactical level. A professional version with realistic parameters is available and is also used by NATO states for the analysis and mission planning of air defence scenarios.

The scenario which is used in this research project contains an enemy aircraft with the mission of flying into a target area over Switzerland. A new anti-aircraft post is attempting to prevent this. The optimal tactic of the approaching aircraft is thus either to fly around the coverage area of the radar or to attack the anti-aircraft post.

An algorithm takes the place of a traditional computer programme and processes predefined steps. The requirement for the algorithm which is to play this game independently is basically as follows: Through trial and error, it should learn independently to win as often as possible. To all intents and purposes, this process can be thought of as human learning. The algorithm builds up experience by trying out various different strategies and selecte.

the most promising one for each situation. These types of algorithms are summarised under the term “reinforcement learning”. Remarkable success has been achieved with this: A software with these types of algorithms beats most, if not all, of the human players in computer games such as Starcraft and even in traditional board games such as Go.

One challenge in the application of reinforcement learning to the computer game “Command: Modern Operations” here lies in particular on the many possible playable actions that can be combined into an unmanageable number of strategies. This is a challenge for the available computing infrastructure. In order to achieve results with the available hardware within a reasonable time, “Artificial Intelligence for Integrated Air Defense” (AI4IAD) of the algorithm “Monte Carlo Tree Search” was tested in the first year of the research project. With this algorithm, the solution space can be restricted to the feasible strategies and thus save computing time. This enabled the remarkable success in the game against the world’s leading Go player Lee Sedol to be achieved, although the game Go had long been considered as unmanageable for artificial intelligence due to its complexity.

However, the algorithm has to complete a high number of games for this purpose. However, as this algorithm is very complex, the connection with the simulation environment proved to be inconsistent. The learning behaviour in simple scenarios proved satisfactory, but this could not be generalised for more demanding scenarios. A further class of algorithms is therefore currently being tested for this type of extended scenario, known as the Deep Q networks. These are easier to handle and learn but less efficient and less stable. Figuratively speaking, these algorithms come up with unusual or unrealistic ideas. They might also be potentially suitable for preparing trainees for unforeseen situations.

Figure 4 shows how artificial intelligence also tries out suboptimal or even futile tactics. In what is known as the convergence phase, only small variations are carried out to arrive at the optimal tactics.

To programme these complex capabilities on a computer, armasuisse S+T has obtained support from the academic community. The Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA) in Lugano enjoys an excellent international reputation in the area of AI research. In addition, armasuisse S+T can look back on many years of successful cooperation with the Institute. For these reasons, IDSIA was commissioned by S+T to support them in the development of a suitable algorithm for optimising air defence.

What’s next?

armasuisse now wants to specifically apply the knowledge gained in air defence with the project AI4IAD and to bring the impact from the laboratory into the field. Further applications of this algorithm already exist in the area of hybrid warfare. The aim is to analyse the added value of current and future technologies in a context relevant for the Swiss Armed Forces – for example, swarms of drones, precision-guided munition or hypersound weapons.

It will be of great importance for operational deployment whether the decisions taken by the algorithms used can also be explained by them. Because two things are important to reach a decision: To know which tactic promises the greatest chance of success – and why. For this reason, armasuisse S+T will in the future explore algorithms which will enable decisions made by artificial intelligence to be better understood and to make them explainable for senior staff and decision-makers. Only once the decisions are transparent, comprehensible and explainable will people be able to trust them. In order to explain decisions even better, this information should be visualised more in the future. Because algorithms generate a vast amount of data which needs to be processed and correspondingly visualised for the persons involved (or concerned). Here, armasuisse S+T will search for the optimum solution for the interaction between humans and machines in further research projects.