Joint Human-Machine Data Exploration

Exploratory data analysis aims at the discovery of knowledge from large-scale data. Thereby, analysts aim to detect and discover both expected patterns but also unexpected aspects in the data. It is an iterative and subjectively controlled process that is usually performed by multiple experts, which further introduces uncertainty in the process. Exploratory data analysis therefore cannot be fully automated. Fully manual exploration is also infeasible as it is extremely time- consuming with todays data scales. We propose joint human-machine data exploration as a new data analysis approach that aims to optimally leverage the joint strengths of machine learning and human visual perception and analytical skills for exploration of large unstructured data. In this basic research project, we develop a dual perspective on exploratory data analysis, bridging the unstructured raw data with the users growing semantic understanding of the data. This dual perspective allows us to introduce a new machine learning approach in the form of an intelligent agent. This intelligent agent tries to incrementally learn the users understanding of the data while they explore the data. Based on this learned understanding, the intelligent agent then performs automated analysis on the data to help the users detect expected patterns and to discover unexpected aspects tailored to their current understanding of the data. The results of this analysis make it possible to optimize the visualization of the data and the interaction techniques to explore the data. In multiple studies with developed software prototypes and human participants, we will investigate how the intelligent agent can learn from the data and one or multiple users, as well as how the user can learn from the data and the intelligent agent in an interactive interplay between knowledge externalization, machine-guided data inspection, questioning, and reframing. The project is a joint collaboration between researchers from TU Wien (Manuela Waldner) and the University of Applied Sciences St. Pölten (Matthias Zeppelzauer), Austria, who contribute and join their complementary expertise on information visualization, visual analytics, and interactive machine learning.