Visual Segmentation and Labeling of Multivariate Time Series (VISSECT)

A highly relevant task in many domains is to find coherent segments of distinguishable events or activities in multivariate time series data. Segmentation is the identification of these coherent subseries within a time series, while the process of labeling assigns a label to each segment, for instance walking in human motion data. Going beyond current segmentation and labeling approaches, this project aims for an interconnected and visual-interactive approach to combine the algorithm selection for segmentation and labeling, the parametrization of these algorithms, and the visualization and exploration of diverse types of uncertainty about the results. (a) A joint system setup, shared sets of data, and task abstractions define a common ground and ensure collaboration throughout the project. We will investigate in the individual aspects with regard to the envisaged interconnections. (a) For an informed and transparent algorithm selection process, we provide visual analytics techniques to explore the selection of adequate segmentation and labeling algorithms, to steer these algorithms, and to guide the users to detect the most adequate algorithms for a particular data set. (b) We ease the parametrization of these algorithms by developing visual analytics techniques for a systematic analysis of the parameter space (many parameters and large value ranges). (c) For exploring and communicating diverse types of uncertainty about the results, we develop visual analytics techniques to assess these types of uncertainty, and allow the investigation of alternative algorithms and parametrizations, aggregated uncertainties, as well as uncertainties about causes and effects. Such a novel strategy requires a comprehensive evaluation. We plan a horizontal as well as a vertical evaluation strategy. With the horizontal evaluation, we will test single visualization and interaction designs that will be developed during the project. With the vertical evaluation, we will provide a summative evaluation of our combined visual analytics approach. Current approaches consider each of these problems separately. However, the tight interconnection of these aspects in our combined visual analytics approach will lead to better results as well as to a deeper understanding of the data and the data generating process; in our case with regard to algorithm selection, parametrization, and involved uncertainty in the segmentation and labelling of multivariate time series.

The goal of VISSECT was to develop new approaches for visually and interactively supporting the segmentation and labeling of multivariate time series (MVTS) through segmentation and labeling algorithms (SLA), which is an important tasks in many domains (like, electrocardiographs in medical domain or human motion tracking). The project team of this German-Austrian collaboration has targeted three particular challenges in the process of segmenting and labeling MVTS: the algorithm selection (TU Darmstadt), the parametrization of algorithms (Universität Rostock), and the uncertainty analysis (TU Wien). The innovative approach of VISSECT was to combine these three challenges within one joint research approach. Accordingly, a major result of VISSECT is the definition of an integrated SLA pipeline for MVTS, which was based on four design goals. (1) the pipeline is general and can be applied to various use cases and application domains. (2) it supports the definition of individual algorithmic routines specific for individual data, users, and tasks. (3) parameters are disclosed and can be defined externally. (4) VISSECT explicitly incorporated concepts to systematically record and propagate uncertainty information with the algorithmic routines and segmentation results. Overall, this pipeline makes the huge design space and possible configurations of SLA pipelines more apparent. VISSECT's main research foci were: (1) advancing the research on algorithm selection, on the interactive coordination of these algorithms, and on user workflows for the SLA pipeline. It is now possible to visually and interactively create SLA pipelines and choose various supervised and unsupervised algorithms that can be analyzed. (2) sophisticated approach for examining parameter settings were designed. The resulting correlation calculation is very flexible and considers arbitrary subspaces of the parameter space. In VISSECT, it is now possible to estimate the parameter influence on a subrange level to support the sampling of the parameter space. (3) exploring uncertainty, which identified several insights. It is now possible to externalize uncertainty from pre-processing (and subsequently data quality), a previous gap in the research on MVTS. This first step led to a better quantification and evaluation of various sources of uncertainty: value, result, aggregation, and cause & effect uncertainty, a clear structure for future research on uncertainty of MVTS. This also helps in understanding how different sources of uncertainty influence the SLA pipeline and the uncertainty visualization. Overall, VISSECT was able to demonstrate significant advances in each of the three challenges of algorithm selection, parametrization of SLA, and uncertainty by a joint reference system of the SLA pipeline. All results have been published at renowned conferences and are available to the wider VA community. The three partners will continue their research in new proposals and initiatives.