RSS feed LinkedIn Watch us on Youtube

Activity title

Employing AI to Federate Sensors in Joint Settings

Activity Reference



System Analysis and Studies

Security Classification




Activity type


Start date


End date



Artificial Intelligence, Data Fusion, Federated Sensor Planning, Joint Intelligence, Machine Learning, Modelling and Simulation, Sensors, Surveillance and Reconnaissance JISR


Joint Intelligence, Surveillance and Reconnaissance (JISR) is often performed by several distinct sensor capabilities, each with their own closed post-processing pipeline and associated command structure. Each individual command carries out distinct mission planning according to its particular information requirements. For instance, the identification of targets may be done in multiple independent ways based on the platforms tasked. Instead of these isolated ISR cycles, a combined planning and mission optimization process is expected to be more effective by actively selecting and allocating multiple complementary sensors to achieve the whole mission objective. Through the JISR concept, NATO is aiming to collect, process and disseminate target information from different systems to relevant users. Research is required to understand how the benefits of multi-sensor fusion can be realized in the JISR concept. The challenge is to assess the combination of multiple sensors and processing methods ahead of time and hence predict the overall effectiveness against the complete set of mission objectives. This points to a problem where the mission goals must be quantified and serve as optimization metrics in a complex and dynamic environment. Fusing information from distinct sensors of similar data type can be found in existing systems, but if the objective is to provide high-level information (e.g. target identification), existing fusion methods may not be sufficient. The problem is exacerbated by the increasing number, diversity and connectivity of available sensor platforms.


The objective of this RTG is to design a system that enables optimal multi-sensor mission planning. This should be based on simulated scenarios, multiple types of sensors, connectivity options and platforms, combined with metrics representing specific mission objectives. This objective includes the need to combine/fuse the resultant data and steps to validate that the mission objectives have been fulfilled with the predicted asset allocations, by using machine learning techniques. Given the dynamics and complexity of the mission planning optimization techniques and artificial intelligence techniques like reinforcement learning will be explored to derive this complex planning.


• Practical optimization methods for sensor management (including reinforcement learning), including methods for defining operationally relevant metrics. • Modeling and simulation for characterization of operational scenarios based on their physical characteristics. • Model validation using high-fidelity simulation. • The use of Machine Learning to process and fuse heterogeneous data from multiple sensors.

Contact Panel Office