RSS feed LinkedIn Watch us on Youtube

Activity title

Computational Imaging and Compressive Sensing for EO/IR Systems

Activity Reference



Sensors & Electronics Technology

Security Classification



Awaiting Publication

Activity type


Start date


End date



Compressive Sensing, Computational Imaging, Data Compression, EO/IR System Design, Modeling, Performance, Signal Processing


Traditional methods of optical design trade optical system complexity for image quality. High quality imagers often require high system complexity. Conventional electro-optical and infrared (EO/IR) systems (i.e. active, passive, multiband and hyperspectral) capture an image by measuring the light incident at each of the millions of pixels in a focal plane array. Recent advances via the combination of optics and image processing have led to potentially significant advances in the capabilities of EO/IR systems. Both computational imaging and compressive sensing have been demonstrated as means to expand the capability of EO/IR systems. Computational imaging has been demonstrated via concepts such as wavefront coding where the depth of field of an imaging system can be extended far beyond normal limitations. It has also been used to demonstrate the reduction in complexity of optical systems. Compressive sensing has emerged as a technique for directly acquiring a compact signal representation without conventional sampling techniques. Compressive sensing has the potential to acquire an image of equal quality to a large format array while providing smaller, cheaper, and lower bandwidth imagers. Compressive sensing also has the potential to drastically challenge the current expectations of what an optical system is supposed to look like. For example, traditional imaging architectures face an intrinsic challenge when collecting multi-dimensional information such as hyperspectral data cubes, which involves a dimensionality mismatch with regard to the classical 2-D measurement space. Standard push-broom or tunable spectral filter architectures trade time dimension in order to capture the data cube. This makes it impossible to simultaneously capture high-speed time varying information. Both computational imaging and compressive sensing have the potential to expand the design trade space for such EO/IR imaging systems. In order to do so, they need to go beyond the traditional imaging components (e.g. lenses, detectors, detector arrays) and architectures. Another example in computational imaging is adaptive task-driven sensing and exploitation for wide area surveillance applications. Active and passive computational imaging systems allow you to allocate your sensing and processing resources to regions of interest in the scene with dynamic content. Translating computational imaging and compressive sensing methods to practical system applications requires tradeoffs between the focal plane array size, shape, format; optical component size, weight, and expense; admissibility of theory in practical systems; and choice of reconstruction method. Performance assessment and modeling of these techniques are critical in determining whether sensors that include such techniques meet operational requirements. Computational imaging and compressive sensing offer the potential for joint optimization of both the optical and algorithmic degrees of freedom toward achieving system level performance and/or SWaP goals. Further assessment of computational imaging and compressive sensing’s impact to an applicable active/passive EO/IR system including automatic image understanding is needed to exploit their potential role in meeting and exceeding operational requirements for military systems. Assessment, design, modeling and data collection efforts would benefit from joint NATO cooperation.


The primary objectives are joint activities to provide common tools for assessing and characterizing computational imaging and compressive sensing techniques for EO/IR imaging sensors and developing design concepts on how to apply them to an imaging system. Research areas include algorithm designs, laboratory assessment, field performance assessment, performance modeling, and conceptual design. Anticipated tools include image sequences allowing assessment of computational imaging and compressive sensing techniques through simulation as well as test set-up configurations for end-to-end sensor performance characterization in the laboratory and field. Efforts will be made to exchange data, algorithms and techniques to establish a common basis for research.


The effort will cover modeling, conceptual design development, laboratory assessment, and field assessment of computational imaging and compressive sensing techniques to determine the performance benefit of these tools for EO/IR tactical sensing. Collected field and laboratory data will be provided to participating nations for the assessment of computational imaging and compressive sensing techniques. Characterization techniques developed will be made available. A Technical Report will be written on the results and provide guidance on how to determine the benefits of compressive sensing techniques. Information may be exchanged with other NATO RTGs such as SET-ET-093 on Performance Evaluation of Compressive Sensing Techniques for Radar and ESM and SET-RTG-190 on Phenomenology and Exploitation of Thermal Hyperspectral Sensing.

Contact Panel Office