Multimodal Explanations for AI-based Multisensor Fusion
DAIS Research Student Dave Braines from IBM Research UK presented his paper ‘Multimodal Explanations for AI-based Multisensor Fusion’ at the NATO SET-262 RSM on Artificial Intelligence for Military Multisensor Fusion Engines in Budapest, Hungary.
The inscrutability of many AI techniques has led to great deal of active research into possible explanations for AI techniques. Within this field, a problem that has received less attention is: what modality of explanation to choose for a particular user and task? For example, many techniques attempt to produce visualizations of the workings of an ML model, e.g., so-called “saliency maps” for a deep neural network, but there may be multiple reasons why this mode of explanation might not be appropriate for a user, including: (i) they may be operating at the edge of the network with a device that is not suited to receiving or displaying such a visualization; (ii) it may not be appropriate for security reasons to send them a visualization derived from the source imagery (e.g., if the location of the camera system is sensitive); (iii) this kind of explanation may be “too low level” for that user’s needs – they may require something more “causal”, for example. One approach that may address all three of these example issues would be to map the explanation from a visualization to a textual rationalization. In this paper we explore this issue of generating explanations in a range of modalities in the context of AI/ML services that operate on multisensor data and show that a “grammar-based” approach that separates atomic explanation-generation and communication actions offers sufficient scope and flexibility to address a set of mission scenarios.
The presentation outlined the proposed architecture and a conceptual model to underpin AI-based explanations, drawing together a number of threads of our previous research. A set of three simple worked examples are shown, do demonstrate the potential use and value of the approach.
This work is part of the DAIS ITA project “Anticipatory Situational Understanding for Coalitions” at CSRI and is a collaboration between Cardiff University and IBM Research UK.