artificial intelligence, C2, human factors, human machine teaming, machine learning, Meaningful Human Control, transparency
This activity addresses an important issue identified in the Specialists Meeting SCI-296 on Autonomy from a System Perspective, held in May 2017 as part of the STO theme devoted to that topic. As noted in the SCI-296 TER, “in many or most cases, it is foreseen that ‘meaningful human control’ (MHC) will be mandated, necessitating the human to maintain awareness and ‘drill down’ on demand”. Responding to this need, the HFM Panel commissioned an exploratory team (HFM-178) to rapidly assess the area from a human-centric perspective. This team came to a consensus as to a working description of MHC, which is “Humans have the ability to make informed choices in sufficient time to influence AI-based systems in order to enable a desired effect or to prevent an undesired immediate or future effect on the environment”. This team also canvased MHC from several dimensions and settled on the need for a dedicated expert-heavy workshop to unpack the most pressing influencing factors. The current proposed activity integrates several research issues emerging from SCI-296, especially those combining humans, (technical) systems, organisation and behaviour.
This activity also directly builds upon work on human-autonomy teaming conducted in HFM-247 on “Human-Autonomy Teaming: Supporting Dynamically Adjustable Collaboration”. In this RTG, experts from 7 countries tracked technology activities, explored novel approaches such as “design patterns”, developed metrics, and prioritized key challenges for future research. Other related NATO activities currently underway include IST-173, IST-157, IST-152, SET-263, SAS-143, SAS-ET-DV, ACT PoW - MUAAR.
Since meaningful human control is deemed to be important for many kinds of automated and (semi)autonomous systems, the term “AI-based systems” is used to encompass all AI-based forms of automation and autonomy, for tasks that are either physical (e.g., unmanned platforms) or informational (e.g., big data analytics, decision support). Given the implications of MHC for the latter application domain, this TAP is also relevant for the STO theme “Big data and AI for military decision making”.
HFM-ET-178 discussed several dimensions associated with MHC, and settled on a subset for increased investigation. These MHC “Themes” have become the organizing focus and objectives of this workshop. These themes are:
• Organizational Considerations of MHC (including team training, agile C2 structures)
• HF guidelines to achieve and maintain MHC for all NATO AI applications
• Systems Engineering methods (including TEV&V for learning systems) to support MHC
• Adversary tactics to counter/undermine MHC (methods and mitigations)
• MHC for complex socio-technical system of systems (emergent properties, MHC propagation)
• Legal, ethical, political, and public perception of MHC over AI-based systems