Self adaptive interfaces and multi-modal human interaction for combat systems
Par Pascale SOULARD
ARTIS FACTA - Ingénierie des Facteurs Humains
published in Undersea Defence Technology, London.1992.
As submarine warfare systems become more powerful and complex, their global efficiency relies more and more upon the quality of the man-system interaction schemes they use. To improve this quality, it is necessary to introduce new interaction media, but also to take into account constraints drawn from human factors in the earliest phases of system design. Such constraints are related to the nature of the media themselves, to the operational context and to the operator individual characteristics.
These principles are embedded within a multi-modal communication system designed to enable the interaction system to choose dynamically and in real time the best suited interaction modes. Keeping track of the interaction events, the system performs a continuous analysis of human activity in order to build and to update dynamically a model of the operator, allowing for user's preferences, habits and reasoning modes. This analysis is conducted on the basis of an exhaustive a priori model of the operator tasks.
The approach enabled us to validate the concept of self-adaptive interfaces, allowing a dynamic adaptation of the interaction logics and of the nature and the form of dialogue media to environmental variations and individual preferences, in order to optimize the global functioning of the system employed.
Introduction : the importance of man-machine communication
Paradoxically the sophistication of tools for interface designer (User Interface Management System, new interaction media as voice input or dataglove) does not increase necessarily the quality of the human computer interaction.
At the opposite, new and multiple interaction media may introduce a problem of choice to the operator; indeed the choice of the best suited interaction media depends on criteria the operator may not have the possibility or the time to evaluate, due to his own faculties and the operational context. Consequently, it is important to check that the operator may not become a restrictive element of the system.
To avoid these problems, man and machine have to be considered as a whole from the earliest phases of system design with the development of advanced multimodal and adaptive Man Machine Interfaces.
This paper presents the concept of advanced interfaces (multimodal and self adaptive) and how we have validated it for a submarine detection system.
The concept of advanced interfaces
Multi-modal user interfaces
A communication (within a multimedia system) is multi-modal if multiple modalities are available for the user, and an input (or output) expression is built up from multiple modalities. A modality may be the mouse, the keyboard, the screen, the tracking ball or more sophisticated modality such as speech recognition or synthesis, gesture recognition ...
Considering this diversification, the communication will gain on adaptability (different possible modalities for a same command: voice or menus) and the application will gain on efficiency (no eyes moving when using input voice in the case of overload situation).
Self Adaptive interfaces
Taking into account both physiological and cognitive human factors enables the system to propose dynamically a set of pertinent data according to the operational context and to the operator mental state. The goal is to facilitate and optimize his task especially in critical situations.
It is important to point out here the difference between self adaptive interfaces and adaptable interfaces.
Adaptable interfaces are defined during the design of the interface taking in consideration only predefined levels of competence.
At the opposite, a system with self adaptive interfaces adapts during run time the nature, the kind of communication devices and the logic of the interactions to the characteristics of the task and to the physiological and cognitive state of the operator.
An important point to insist on is that, in our ASW application, operators can have varying levels of skill, preferred strategies when performing a task with varying levels of demand. Indeed, during the exploitation of multi-sensor detection or decision making system the perform task varies according to the mission. For instance, during transit phase the demand level of the task is relatively low and the operator is in an undervigilance state. At the opposite during attack or tracking phase, stress is very important and the operator has to face an higher workload. Those remarks lead us to use self adaptive interfaces for such system.
Moreover the operator characteristics rely more on different individual behaviour (habits, preferences) than demand levels. All the operators are experts in their domain, but they use neither the same reasoning nor the same tools to perform a task.
SAITeR : an application of advanced interfaces for combat systems
SAITeR (Séquencement d'Activités Intelligent en Temps Réel, i.e. Intelligent Process Scheduling) is an application for submarine designed and developed in The Advanced Research Department of TS.ASM Arcueil. It automatically performs a complete scheduling of TMA (Target Motion Analysis) tasks , each task running a specific data processing algorithm whose triggering depends on operational and technical context evolution. The objective is to provide the position and kinematics of detected vessels as accurately as possible. In order to achieve this objective, SAITeR chooses the "well-matching" algorithms to trigger at the "well-matching" time with relevant information from the "well-matching" vessels (analysing the quality and the quantity of information).
A complex system
SAITeR consists of two parts:
Different elements such as the (A) part screen overload, the relevance of the information display on vessels (the information order, its accuracy), the succession of necessary tasks to run an algorithm in the ATF part (section of track selection, tacking into account past results), lead to evaluate the necessary part (impact) of man-system communication to optimize the global functioning of the man-machine system.
SAITeR and self adaptive interfaces
As noticed above, a dynamic adaptation of the interaction logics and of the information quality has to be allowed.
Some simples ideas enable to illustrate the concept of self adaptive interfaces of SAITeR:
SAITeR and multimodal communication
The development of SAITeR naturally leads to analyse globaly the operator workstation. Even if not necessary, the diversification of interaction media can improve the conditions of work and the operator performances. Indeed, the results of a study done by an ergonom raise up constraints on the existing workstation:
These disadvantages lead us to imagine a multimodal workstation composed of a touch entry screen (in place of some buttons), a voice input to keep eyes on the screen during some command, the speech synthesis under some conditions such as the use of headphone to reduce the ambient noise or the use of short messages.
The generic architecture
We have been working on advanced Human Interface and on Cognitive Science for two years in a THOMSON-CSF Strategic project called "Poste de Travail Intelligent" (Intelligent Workstation). In this project we have defined a generic architecture which enables to build multimodal and self adaptive interfaces for operational applications.
As we have seen above, taking into account such capabilities requires a design for the Man Machine Interface based on a modelling of operational task, a dynamic modelling of the operator dynamically taking into account the operational context and by a previous analysis of the activity from an ergonomic point of view. So we have embedded this approach within a multimodal communication system based on a distributed knowledge-base design and we have validated it with SAITeR, integrating a recognition/synthesis speech system in addition to the mouse and keyboard.
This generic architecture is composed of three main modules:
These concepts have been validated on a specific application. Yet, many applications can be found in naval warfare, in which the concept of advanced interface improves the performance of the system.
To conclude we wish to emphasize the necessary cooperation between experts from different disciplines (advanced information processing, ergonomics, cognitive psychology and operational people) in order to design interactive systems where the solving of specific problems is deleguated to the expert from the concerned domain.
1 AMALBERTI, R. (1991). Modèles de raisonnement et ergonomie cognitive. Actes des Entretiens Science et Défense (pp. 317-328). Paris: Dunod.
2 BARTHET, M. F. (1988). Logiciels interactifs et ergonomie. Paris: Dunod.
3 COUTAZ, J. (1990b). Interfaces homme-ordinateur. Paris: Dunod.
4 HOLLNAGEL, E. (1988). Mental models and model mentality. In L. P. Goodstein, H. B. Andersen, & S. E. Olsen (Eds), Tasks, errors and mental models ( pp. 261-268). London: Taylor & Francis.
5 MANCINI, G. (1988). Modelling humans and machines. In L. P. Goodstein, H. B. Andersen, & S. E. Olsen (Eds), Tasks, errors and mental models (pp. 278-292). London: Taylor & Francis.
6 PINSKY, L. (1988). Cours d'action et activité. XXIV congrès de la SELF, Paris, pp. 29-45.
7 RASMUSSEN, J., 1986, A Cognitive Engineering Approach to the Modelling of Decision Making and its organization, Riso-M-2589 (Roskilde, Denmark: Riso National Laboratory).
8 RICHARD, J. F. (1990). Les activités mentales. Paris: Armand Colin.
9 SEBILLOTTE, S. (1991). Décrire des tâches selon les objectifs des opérateurs (Rapport technique ndeg. 125), INRIA: Rocquencourt.
10 WOODS, D. D. & ROTH, E. M. (1988). Aiding human performance II: from analysis to support systems. Le travail humain, 51, pp. 139-172.
haut de page