Explainable AI (XAI) seeks to disclose how an AI system arrives at its outcomes. But the nature of the disclosure depends in part on who needs to understand the AI and the available explanation modalities (e.g., verbal and visual). Users’ preferences regarding explanation modalities might differ, as some might prefer spoken explanations compared to visual ones. However, we argue for broadening the explanation modalities, to consider also tangible and physical forms. In traditional product design, physical forms have mediated people’s interactions with objects; more recently interacting with physical forms has become prominent with IoT and smart devices, such as smart lighting and robotic vacuum cleaners. But how tangible interaction can support AI explanations is not yet well understood.

In this second studio proposal on Graspable AI (GAI) we seek to explore design qualities of physical forms as an explanation modality for XAI. We anticipate that the design qualities of physical forms and their tangible interactivity can not only contribute to the explainability of AI through facilitating dialogue, relationships and human empowerment, but they can also contribute to critical and reflective discourses on AI. Therefore, this proposal contributes to design agendas that expand explainable AI into tangible modalities, supporting a more diverse range of users in their understanding of how a given AI works and the meanings of its outcomes.

Computational systems are becoming increasingly smart and automated. Artificial intelligence (AI) systems perceive things in the world, produce content, make decisions for and about us, and serve as emotional companions. From music recommendations to higher-stakes scenarios such as policy decisions, drone-based warfare, and automated driving directions, automated systems affect us all.

But researchers and other experts are asking, How well do we understand this alien intelligence? If even AI developers don’t fully understand how their own neural networks make decisions, what chance does the public have to understand AI outcomes? For example, AI systems decide whether a person should get a loan; so what should—what can—that person understand about how the decision was made? And if we can’t understand it, how can any of us trust AI?

The emerging area of explainable AI (XAI) addresses these issues by helping to disclose how an AI system arrives at its outcomes. But the nature of the disclosure depends in part on the audience, or who needs to understand the AI. A car, for example, can send warnings to consumers (“Tire Pressure Low”) and also send highly technical diagnostic codes that only trained mechanics can understand. Explanation modality is also important to consider. Some people might prefer spoken explanations compared to visual ones. Physical forms afford natural interaction with some smart systems, like vehicles and vacuums, but whether tangible interaction can support AI explanation has not yet been explored.

In the summer of 2020, a group of multidisciplinary researchers collaborated on a studio proposal for the 2021 ACM Tangible Embodied and Embedded (TEI) conference. The basic idea was to link conversations about tangible and embodied interaction and product semantics to XAI. Here, we first describe the background and motivation for the workshop and then report on its outcomes and offer some discussion points.

This article investigates the event-triggered adaptive fuzzy output feedback setpoint regulation control for the surface vessels. The vessel velocities are noisy and small in the setpoint regulation operation and the thrusters have saturation constraints. A high-gain filter is constructed to obtain the vessel velocity estimations from noisy position and heading. An auxiliary dynamic filter with control deviation as the input is adopted to reduce thruster saturation effects. The adaptive fuzzy logic systems approximate vessel's uncertain dynamics. The adaptive dynamic surface control is employed to derive the event-triggered adaptive fuzzy setpoint regulation control depending only on noisy position and heading measurements. By the virtue of the event-triggering, the vessel's thruster acting frequencies are reduced such that the thruster excessive wear is avoided. The computational burden is reduced due to the differentiation avoidance for virtual stabilizing functions required in the traditional backstepping. It is analyzed that the event-triggered adaptive fuzzy setpoint regulation control maintains position and heading at desired points and ensures the closed-loop semi-global stability. Both theoretical analyses and simulations with comparisons validate the effectiveness and the superiority of the control scheme. 

In this paper, we investigate the dynamic task scheduling problem with stochastic task arrival times and due dates in the UAV-based intelligent on-demand meal delivery system (UIOMDS) to improve the efficiency. The objective is to minimize the total tardiness. The new constraints and characteristics introduced by UAVs in the problem model are fully studied. An iterated heuristic framework SES (Stochastic Event Scheduling) is proposed to periodically schedule tasks, which consists of a task collection and a dynamic task scheduling phases. Two task collection strategies are introduced and three Roulette-based flight dispatching approaches are employed. A simulated annealing based local search method is integrated to optimize the solutions. The experimental results show that the proposed algorithm is robust and more effective compared with other two existing algorithms.

This paper investigates the tracking control problem of marine surface vessels (MSVs) in the presence of uncertain dynamics and external disturbances. The facts that actuators are subject to undesirable faults and input saturation are taken into account. Benefiting from the smoothness of the Gaussian error function, a novel saturation function is introduced to replace each nonsmooth actuator saturation nonlinearity. Applying the hand position approach, the original motion dynamics of underactuated MSVs are transformed into a standard integral cascade form so that the vector design method can be used to solve the control problem for underactuated MSVs. By combining the neural network technique and virtual parameter learning algorithm with the vector design method, and introducing an event triggering mechanism, a novel event-triggered indirect neuroadaptive fault-tolerant control scheme is proposed, which has several notable characteristics compared with most existing strategies: 1) it is not only robust and adaptive to uncertain dynamics and external disturbances but is also tolerant to undesirable actuator faults and saturation; 2) it reduces the acting frequency of actuators, thereby decreasing the mechanical wear of the MSV actuators, via the event-triggered control (ETC) technique; 3) it guarantees stable tracking without the a priori knowledge of the dynamics of the MSVs, external disturbances or actuator faults; and 4) it only involves two parameter adaptations--a virtual parameter and a lower bound on the uncertain gains of the actuators--and is thus more affordable to implement. On the basis of the Lyapunov theorem, it is verified that all signals in the tracking control system of the underactuated MSVs are bounded. Finally, the effectiveness of the proposed control scheme is demonstrated by simulations and comparative results. 

Nearly every real-world deployment of machine learning models suffers from some form of shift in data distributions in relation to the data encountered in production. This aspect is particularly pronounced when dealing with streaming data or in dynamic settings (e.g. changes in data sources, behaviour and the environment). As a result, the performance of the models degrades during deployment. In order to account for these contextual changes, domain adaptation techniques have been designed for scenarios where the aim is to learn a model from a source data distribution, which can perform well on a different, but related target data distribution. In this paper we introduce a variational autoencoder-based multi-modal approach for the task of domain adaptation, that can be trained on a large amount of labelled data from the source domain, coupled with a comparably small amount of labelled data from the target domain. We demonstrate our approach in the context of human activity recognition using various IoT sensing modalities and report superior results when benchmarking against the effective mSDA method for domain adaptation.

Interactive Machine Learning (ML) has the potential to lower the manual labelling effort needed, as well as increase classification performance by incorporating a human-in-the loop component. However, the assumptions made regarding the interactive behaviour of the human in experiments are often not realistic. Active learning typically treats the human as a passive, but always correct, participant. Machine teaching provides a more proactive role for the human, but generally assumes that the human is constantly monitoring the learning process. In this paper, we present an interactive online framework and perform experiments to compare active learning, machine teaching and combined approaches. We study not only the classification performance, but also the effort (to label samples) and attention (to monitor the ML system) required of the human. Results from experiments show that a combined approach generally performs better with less effort compared to active learning and machine teaching. With regards to attention, the best performing strategy varied depending on the problem setup.

In this paper, we study explanations in a setting where human capabilities are in parity with Machine Learning (ML) capabilities. If an ML system is to be trusted in this situation, limitations in the trained ML model’s abilities have to be exposed to the end-user. A majority of current approaches focus on the task of creating explanations for a proposed decision, but less attention is given to the equally important task of exposing limitations in the ML model’s capabilities, limitations that in turn affect the validity of created explanations. Using a small-scale design experiment we compare human explanations with explanations created by an ML system. This paper explores and presents how the structure and terminology of scientific explanations can expose limitations in the ML models knowledge and be used as an approach for research and design in the area of explainable artificial intelligence.

Quantifying the number of occupants in an indoor space is useful for a wide variety of applications. Attempts have been made at solving the task using passive infrared (PIR) motion sensor data together with supervised learning methods. Collecting a large labeled dataset containing both PIR motion sensor data and ground truth people count is however time-consuming, often requiring one hour of observation for each hour of data gathered. In this paper, a method is proposed for generating such data synthetically. A simulator is developed in the Unity game engine capable of producing synthetic PIR motion sensor data by detecting simulated occupants. The accuracy of the simulator is tested by replicating a real-world meeting room inside the simulator and conducting an experiment where a set of choreographed movements are performed in the simulated environment as well as the real room. In 34 out of 50 tested situations, the output from the simulated PIR sensors is comparable to the output from the real-world PIR sensors. The developed simulator is also used to study how a PIR sensor’s output changes depending on where in a room a motion is carried out. Through this, the relationship between sensor output and spatial position of a motion is discovered to be highly non-linear, which highlights some of the difficulties associated with mapping PIR data to occupancy count.