RSS feed LinkedIn Watch us on Youtube

Activity title

Assessment of EO/IR Compressive Sensing and Computational Imaging Systems

Activity Reference



Sensors & Electronics Technology

Security Classification




Activity type


Start date


End date



AI for reconstruction, coded aperture imaging, detection and imaging, lensless imaging, light field camera, nonlineofsight sensing, range imaging, Single pixel camera, spectral aspects, wavefront coding


Conventional electro-optical and infrared (EO/IR) systems (i.e. active, passive, multiband and hyperspectral) capture an image by optically focusing the incident light at each of the millions of pixels in a focal plane array. The optics and the focal plane are designed to efficiently capture desired aspects (like spectral content, spatial resolution, depth of focus, polarization, etc.) of the scene. Computational imaging refers to image formation techniques that use digital computation to recover an image from an appropriately multiplexed or coded light intensity of the scene. In this case, the desired aspects of the scene can be selected at the time of image reconstruction which allows greater operational flexibility to an EO/IR system. Compressive sensing involves capturing a smaller number of specifically designed measurements from the scene to computationally recover the image or task specific scene information. Compressive sensing has the potential to acquire an image with equivalent information content to a large format array while using smaller, cheaper, and lower bandwidth components. Moreover, the data acquisition can be sequenced and designed to capture task specific and mission relevant information guided by the scene content with more flexibility. However, the benefits of compressive sensing and computational imaging do not come without compromise, such as increased field-of-view with a multiplexed imager at the expense of decreased signal-to-noise ratio or increased spatial resolution with the single-pixel camera at the expense of dynamic range and motion smear. NATO SET-232 has made a first step in investigating the promise of computational imaging and compressive sensing for EO/IR systems, but now need to quantify performance during a field trial in a follow-on effort. In the proposed ET and RTG, candidate computational imaging and compressive sensing EO/IR systems will undergo an end-to-end performance assessment of relevant military tasks to characterize and quantify the benefits and limitations of this technology.


Although computational imaging and compressive sensing are relatively young technologies, many significant results have been achieved in recent years that must be reviewed and analyzed to ensure that the overall operational benefits are realized. The scientific objective of this ET and the subsequent RTG is to conduct an end-to-end performance assessment of a military relevant task using candidate computational imaging and compressive sensing EO/IR systems. This includes: - Selecting candidate computational imaging and compressive sensing systems o Multiplexed imaging system, compressive sensing ranging system, single pixel camera, non-line-of-sight imaging system, compressive sensing hyperspectral imaging system, etc. - Defining military relevant tasks - Conduct joint field trials - Apply assessment metrics o Computational complexity, sparsity assumption, signal-to-noise ratio, resolution, image quality metrics, latency, etc. - Soldier/Algorithm Performance Testing o False alarm rate, identification rate, detection, recognition, and identification range, acquisition time, etc. - Performance Modeling - SWAP-C considerations o Reliability, time-to-failure, new form-factor applications and platform compatibility


The ET and the subsequent RTG will focus on performance assessment of a subset of the following technologies: • Single pixel camera for different spectral bands: visual, short wave infrared (SWIR), thermal IR, THz. • Non-line-of-sight sensing: detection and imaging • Coded aperture: wavefront coding, coded aperture imaging, spectral compressive sensing and lensless imaging • Time domain: compressive sensing range imaging • Multiplexing: microscanning super resolution, optical multiplexing, multispectral super resolution • Data Compression: encryption, low bandwidth • Other technologies: light field camera, spectral mosaic patterns, artificial intelligence for reconstruction.

Contact Panel Office