RSS feed LinkedIn Watch us on Youtube

Activity title

Multi-Level Fusion of Hard and Soft Information

Activity Reference

IST-132 (AI2S)


Information Systems Technology

Security Classification




Activity type


Start date


End date



Hard Fusion, Information Fusion, Intelligence, Metadata, Multi-modal Sensor Fusion, Sensor Data Fusion, Soft Fusion


Up to now there is no smooth and integrated data and information processing within the overall ISTAR chain. Situation awareness on device-based data, mainly built from the products of algorithmic sensor data fusion and used in near real-time applications is poorly connected to high level information fusion providing the military leader with a situation awareness mainly based on soft information. The poor connection between these communities is caused by the lacking of a unified approach to processing hard data from reconnaissance and soft information, typically from HUMINT, in a holistic manner. This results in loss of timeliness, adequacy, preciseness and comprehensiveness of situation awareness as bases for a common operational picture and a loss of opportunities in own operations. Linking near real-time fusion of hard data with soft information fusion for tactical and operational intelligence will not only improve situational awareness, threat detection and impact assessment but also contribute to all those functions in the military C2 cycle which are closely related to these intelligence products, e.g., collection and mission planning and resource management.


The objective of this task group is to conduct research on data and information fusion incorporating structured and unstructured human generated information (soft data) facing the complex relevant information context and heterogeneous data structures from various civilian and military organisations and sources. The results of the previous STO Task Group IST-106 on Information Filtering and Multi-Source Information Fusion will serve as a basis for these investigations, especially the provided data set of a live experiment on a surveillance scenario comprised of hard and soft data. The main objectives will be to demonstrate the usefulness of a Controlled Language (BML) as a mechanism to facilitate integrated exploitation of data and information from different type of sources and on different levels of fusion, and to improve threat detection and modelling, including dealing with uncertainty and detection of anomalous behaviour as a result of the holistic data and information processing.


a. Further explore potential of BML to be a common framework to uniformly results of hard data fusion in a way that can be used for soft information fusion and vice versa. b. Further develop structured representations of lower-level situational element for enhanced integration in higher level information processing using situation context information c. Further develop transformation of soft data into computationally exploitable information with robust quality measures exploring various mathematical formalisms: e.g. computational linguistics, grey information theory, Bayesian, DS, DSmT, fuzzy logic), improving the usability of soft information.

Contact Panel Office