Information visualization for ground situation awareness

Project summary

Ground situation analysis is characterised by large amounts of information that typically need be to be handled not only by machines, but also visualized for human decision-makers, who typically are under time pressure. Hence, this project mainly concerns the situation level of the JDL model, and in particular the human-computer interaction aspect. The problem to be studied includes how to present situation analysis information to a user/decision-maker in such a way that it can be effectively assessed and fused in a semi-automatic fashion. The problem domain is typically characterized by the need for rapid decision making and the presence of many different sources of information. This means that a part of the problem is to focus attention on the most important pieces of information. It is also common that a particular situation can be viewed from different perspectives (e.g., information about targets with respect to their geographical position, fuel situation, capability, etc.) depending on which role the user of the system has.

The focus of this project is to investigate how semi-automatic means for information fusion can be integrated with automatic means in a resource effective way. This is particularly important when there are limitations on the visualization resources. A further goal is to compare and contrast the effects of availability of different resources in a command center and in the field. 

Research question

The general research question to be studied is how fused situation analysis information can be presented to a user in such a way that it can be
assessed and used in a semi-automatic fashion to effectively support decision-making. This includes issues such as:

• Study and development of interaction methods that allow the involvement of the user in the fusion process, data exploration and knowledge discovery (e.g. how to input user's knowledge or experience into the system);
• Visualization of large data sets: patterns, trends, changes over time, outliers, exceptions, anomalies, relationships, correlations, clusters, groups, context, etc;
• Visualization of uncertainty, reliability, completeness and quality of information;
• Visualize different levels of abstraction or granularity  (in time and space).

Relevance to information fusion

Visualization of fused information for human decision-makers is a crucial component of application domains like the ones studied here, where critical decisions need to be taken by humans, often under time pressure. Military applications in a network-centric warfare scenario are a typical example, but this also applies to many types of civilian operations, e.g. dealing with different types of catastrophes.

This project will be able to contribute to the IF research program’s common goals by furthering our understanding of the requirements and possibilities of interactive, user- and situation-dependent information fusion visualization at different levels of abstraction. Thus this project will contribute in particular to the common-goals framework project (cgi1) that looks at the cognitive constraints on information fusion and its use in decision support, but also indirectly to several other projects in which visualization of fused information for decision-makers is a crucial element.

Approach

A number of concrete scenarios, military and civilian, had been developed, in close cooperation with SAAB Microwave Systems (including border control, surveillance of sea areas and intrusion in a military camp). The simulated data is used to test data mining and visualization techniques. Currently, a scenario using a simulation tool, Stage, is being developed. The scenario will be used to test visual analytics methods and techniques that can be applicable in information fusion

This project will additionally focus on interactivity of visualization, i.e. giving users the possibility of adapting the mode of visualization actively rather than being passive recipients of visual displays. This project will as well apply recent developments in information display technology to the problem of enhancing situation awareness (SA).

Related work

Exploring, analyzing and finding the relevant information in vast amounts of multidimensional sensor data is a complex task. Data mining techniques can filter and extract valuable patterns. The need to tightly include the human in the exploration process is recognized by many authors. Visual data mining focuses on integrating the user in the knowledge discovery process using effective and efficient visualization techniques and interaction capabilities.

An emerging research area in the past years is visual analytics. Visual analytics is defined as analytical reasoning supported by highly interactive visual interfaces. Contributions in this area integrate information visualization, interaction and computational analysis in order to transform massive data into knowledge. When the data analyzed is space related, models, methods and tools presented in geovisual analytics are worth considering. Related to information fusion is also work on visualization of uncertainty.

Members

Göran Falkman, PhD (Project Leader).
Tom Ziemke, professor.
Maria Riveiro, guest lecturer.

Publications

Riveiro, M. (2011) Visual analytics for maritime anomaly detection. Doctoral Dissertation. Örebro Studies in Technology, 46. ISBN 978-91-7668-782-6.  http://oru.diva-portal.org/smash/record.jsf?pid=diva2:381336&searchId=null

Riveiro, M. and Falkman, G. (2011) Empirical evaluation of visualizations of normal behavioral models for supporting maritime anomaly detection. Accepted for presentation and publication at GeoViz: Linking Geovisualization with Spatial Analysis and Modeling, March 10–11, Hamburg, Germany.

Riveiro, M. and Falkman, G. (2011) The role of visualization and interaction in maritime anomaly detection. In: Chung Wong, P. et al. (Eds.) Visualization and Data Analysis 2011. Proceedings of SPIE-IS&T Electronic Imaging, 23-27 January, San Francisco, CA, USA. Volume 7868, pp. 78680M 1–12.

Riveiro, M. and Falkman, G. (2010) Evaluating the usability of normal behavioral models visualizations for analytical reasoning. In Banissi, E., Sarfraz, M., and Huang, M.L. (Eds.) Proceedings of the 2010 7th International Conference on Computer Graphics, Imaging and Visualization (CGIV 2010), 7–10 August 2010, Sydney, Australia, pp 179–185. IEEE Computer Society.

Riveiro, M. and Falkman, G. (2010) Supporting the analytical reasoning process in maritime anomaly detection: Evaluation and experimental design. In Banissi, E. et al. (Eds.) Proceedings of the 2010 14th International Conference on Information Visualisation (IV 2010), 26–29 July 2010, London, UK, pp 170–178. IEEE Computer Society.

Riveiro, M. and Falkman, G. (2009) Interactive Visualization of Normal Behavioral Models and Expert Rules for Maritime Anomaly Detection. Accepted for publication and presentation at The 6th International Conference on Computer Graphics, Imaging and Visualization (CGIV09), IEEE. 11–14 August 2009, Tianjin, China.

Riveiro, M., Falkman, G., Ziemke, T., and Kronhamn, T. (2009) Reasoning about anomalies: a study of the analytical process of detecting and identifying anomalous behavior in maritime traffic data. In Tolone, W.J. and Ribarsky, W. (Eds.) Visual Analytics for Homeland Defense and Security. Proceedings of SPIE Defense, Security, and Sensing 2009, 13–17 April 2009, Orlando, FL, USA. SPIE Volume 7346, 73460A. DOI: 10.1117/12.818117.

Riveiro, M., Falkman, G., Ziemke, T., and Warston, H. (2009) VISAD: an interactive and visual analytical tool for the detection of behavioral anomalies in maritime traffic data. In Tolone, W.J. and Ribarsky, W. (Eds.) Visual Analytics for Homeland Defense and Security. Proceedings of SPIE Defense, Security, and Sensing 2009, 13–17 April 2009, Orlando, FL, USA. SPIE Volume 7346, 734607. DOI: 10.1117/12.817819.

Riveiro, M., Falkman, G., and Ziemke, T. (2008) Visual Analytics for the Detection of Anomalous Maritime Behavior. In Banissi, E. et al. (Eds.) Proceedings of the 12th IEEE International Conference on Information Visualisation (IV’08), London, UK, July 9–11 2008, pp 273–279. IEEE.

Riveiro, M., Falkman, G., and Ziemke, T. (2008) Improving maritime anomaly detection and situation awareness through interactive visualization. In Proceedings of the 11th IEEE International Conference on Information Fusion (FUSION 2008), Cologne, Gemany, June 30–July 3, 2008, pp 47–54. IEEE.

Riveiro, M., Johansson, F., Falkman, G., and Ziemke, T. (2008) Supporting Maritime Situation Awareness Using Self Organizing Maps and Gaussian Mixture Models. In Holst, A., Kreuger, P., and Funk, P. (Eds.) Tenth Scandinavian Conference on Artificial Intelligence. Proceedings of SCAI 2008. Frontiers in Artificial Intelligence and Applications 173, pp 84–91. IOS Press.

Niklasson, L., Riveiro, M., Johansson, F., Dahlbom, A., Falkman, G., Ziemke, T., Brax, C., Kronhamn, T., Smedberg, M., Warston, H., and Gustavsson, P.M. (2008) Extending the Scope of Situation Analysis. In Proceedings of the 11th IEEE International Conference on Information Fusion (FUSION 2008), Cologne, Gemany, June 30–July 3, 2008, pp 454–461. IEEE.

Niklasson, L., Riveiro, M., Johansson, F., Dahlbom, A., Falkman, G., Ziemke, T., Brax, C., Kronhamn, T., Smedberg, M., Warston, H., and Gustavsson, P.M. (2007) A Unified Situation Analysis Model for Human and Machine Situation Awareness. In Koschke, R., Herzog, O., Rödiger, K.-H., and Ronthaler, M. (Eds.) Trends, Solutions, Applications. Proceedings of SDF 2007. LNI P-109, pp 105–110. Köllen Druck & Verlag.

Riveiro, M. (2007) Evaluation of Uncertainty Visualization Techniques for Information Fusion. Proceedings of the 10th International Conference on Information Fusion (ICIF ‘07), Québec, Canada, July 9–12, 2007, pp 1–8. IEEE

Riveiro, M. (2007) Cognitive Evaluation of Uncertainty Visualization Methods for Decision Making. Symposium on Applied Perception in Graphics and Visualization (APGV 2007), Tübingen, Germany, July 25–27, pp 133. ACM.

Schäfer, B., Bergfeldt N., Riveiro M., and Ziemke, T. (2007) Evolution of Tool Use Behavior. Proceedings of the 2007 IEEE Symposium on Artificial Life, pp 31–38. IEEE.