A virtual safety advisor for ATM

1. Objective

The introduction of automation is considered to be a key step in bridging the expected gap between capacity and demand in the harmonization of the European Airspace (SESAR). The allocation of automated functions to a system is often assumed to be associated with an increase in efficiency and safety performance. The rationale behind this reasoning is that the technical systems will be able to perform more reliable than a human operator regarding specific functions.
Research indicates, that while the increase in safety performance holds true for specific functions of the automation, unintended side-effects of the automation will occur, if the expected functioning of the automation is based on wrong assumptions regarding the way humans actually actively produce safety (e.g. Bainbridge, 1983; Hollnagel & Woods, 2005; Metzger & Parasuraman, 2005).
These unintended side-effects largely result from the fact that while the automation removes the negative aspects of human performance (the errors) it does not maintain the positive aspects of human performance (e.g. dealing with unanticipated situations or beyond design-base scenarios).
Thus, the method to be developed in the context of this phd-activity will help in gaining a more complete understanding on the way safety is produced by the people actually conducting the work. More specifically, it will show how humans actually structure their tasks according to approximate adaptations (trade-offs and heuristics), resulting in performance variability (EUROCONTROL, 2009). To achieve this, a computational method will be developed based on dynamic performance data with the aim to demonstrate how human operators prioritize task-relevant information during work and how these prioritization relate to necessary task-related approximate adaptations (trade-offs and heuristics).

2. Methodology

The empirical data collected in the studies consists of dynamic visual behavior recorded with eye-tracking devices. Thus, detailed, time-bound analyzes of the visual prioritization of information sources can be conducted and set into relation with trade-offs and heuristics. To this end, the eye-tracking data will be triangulated with other methods (questionnaires, performance tests, semi-structured interviews and selected other dynamic performance measurements).
Based on the triangulated eye-tracking data, indicators sensitive to the approximate adaptations (trade-offs, heuristics) behind performance variability will be derived based on the current state of the literature and empirical data.
To this end, two empirical field studies have been conducted in order to develop the different indicators:
• Field study of real control tasks of heavy machinery
• Field study of complex decision-making tasks under time-pressure by professionals in the medical domain (anesthesia simulator).
In a second step, the indicators will be validated for the aviation domain in a large scale evaluation of human performance in real long-haul flights using eye-tracking data and other methods for data triangulation.

3. Results/Findings

Indicators sensitive for the information prioritization associated with approximate adaptations have been developed.
Based on these indicators, a time-based and changing model of the information prioritization can be computed, that shows which information sources the human operator currently prioritizes during the conduct of a specific task. This information can in turn be used to determine if there is a change in heuristics or trade-offs by the operator for a specific task.
Furthermore, a markerless system for the analysis of eye-tracking data in the aviation domain has been developed. This enables the application of the eye-tracking device in technical environments, where the placement of markers would interfere with the conduct of normal work (e.g. flight deck).

4. Implications for Research

A fundamental challenge with the introduction of automation consists in guaranteeing that the safety targets are met, that is, that the systems stays at least as reliable as it was before the introduction of the new technology (Sträter et al., 2012).
The method that is developed in the context of this phd activity, makes it possible to systematically capture and analyze dynamic performance data for specific tasks in real-life settings (based on triangulated eye-tracking data).
Furthermore, by means of a computational model, continuous representations of the prioritizations of information sources can be computed that enable the identification of adaptive processes (trade-offs and heuristics) necessary for system functioning.
These adaptive processes can in turn be used for further analyses by means of bounded-rationality agent-based modeling (ComplexWorld, 2012), task-models addressing performance variability (e.g.FRAM, Hollnagel, 2012; EUROCONTROL, 2009) or in terms of decision-making regarding automation of functions (Does the automation assist the approximate behavior of human operators? Does the automation ensure that the positive contribution of the approximate behavior stays in the system is the function is completely automated?).  

5. References

Arenius, M., & Sträter, O. (2013). Resilience Engineering in Railways – Results from a systemic accident and event analysis in German railways. Presented at the ESREL 2013, Amsterdam.
Arenius, M., Tewes, T., Harif, A., Athanassiou, G., & Sträter, O. (2011). Arbeitsbericht Berücksichtigung menschlicher Faktoren bei der Systemgestaltung im Eisenbahnwesen „HUMAN FACTORS“: Abschlussbericht - Hauptversuch (No. 4). Munich: Deutsche Bahn.
Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775–779. doi:10.1016/0005-1098(83)90046-8
ComplexWorld. (2012). The ComplexWorld Position Paper (p. 70). Retrieved from http://www.complexworld.eu/wp-content/uploads/2012/03/ComplexWorld-Posit...
EUROCONTROL. (2009). A White Paper on Resilience Engineering for ATM. Eurocontrol. Retrieved from http://www.eurocontrol.int
Hollnagel, E. (2012). FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-technical Systems. Ashgate Publishing, Ltd.
Hollnagel, E., & Woods, D. D. (2005). Joint cognitive systems: foundations of cognitive systems engineering. Boca Raton, FL: Taylor & Francis.
Metzger, U., & Parasuraman, R. (2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47(1), 35–49.
Sträter, O., Dolezal, R., Arenius, A., & Athanassiou, G. (2012). Status and Needs on Human Reliability Assessment of Complex Systems. SRESA Journal of Life Cycle Reliability and Safety Engineering, 1(1), 22–43.