The recent expansion of metrification on a daily basis has led to the production of massive quantities of data, and in many cases, these collected metrics are only useful for knowledge building when seen as a full sequence of data ordered by time, which constitutes a time series. To find and interpret meaningful behavioral patterns in time series, a multitude of analysis software tools have been developed. Many of the existing solutions use annotations to enable the curation of a knowledge base that is shared between a group of researchers over a network. However, these tools also lack appropriate mechanisms to handle a high number of concurrent requests and to properly store massive data sets and ontologies, as well as suitable representations for annotated data that are visually interpretable by humans and explorable by automated systems.

The goal of the work presented in this dissertation is to iterate on existing time series analysis software and build a platform for the collaborative analysis of massive time series data sets, leveraging state-of-the-art technologies for querying, storing and displaying time series and annotations. A theoretical and domain-agnostic model was proposed to enable the implementation of a distributed, extensible, secure and high-performant architecture that handles various annotation proposals in simultaneous and avoids any data loss from overlapping contributions or unsanctioned changes. Analysts can share annotation projects with peers, restricting a set of collaborators to a smaller scope of analysis and to a limited catalog of annotation semantics.

Annotations can express meaning not only over a segment of time, but also over a subset of the series that coexist in the same segment. A novel visual encoding for annotations is proposed, where annotations are rendered as arcs traced only over the affected series' curves in order to reduce visual clutter.

Moreover, the implementation of a full-stack prototype with a reactive web interface was described, directly following the proposed architectural and visualization model while applied to the HVAC domain. The performance of the prototype under different architectural approaches was benchmarked, and the interface was tested in its usability. Overall, the work described in this dissertation contributes with a more versatile, intuitive and scalable time series annotation platform that streamlines the knowledge-discovery workflow.

Outline

1. Introduction overviews the various concepts surrounding time series analysis, time series visualization techniques, digital annotations (and how these have been used in time series analysis), and distributed systems;

2. State of the art presents an in-depth overview of state of the art methodologies and technologies currently applied to analysis, storage and visualization of time series and annotations;

3. Proposed Model proposes a blueprint for the development of a time series analysis and annotation platform at the theoretical level, describing an ideal architecture and visualization tool for handling both time series and ontology data;

4. Implementation iterates on the blueprint proposed in Chapter 3 by implementing and describing a working prototype for a collaborative time series analysis architecture and web application, listing the specific tools that were employed for it and the techniques that allowed further optimized usage of state of the art technologies to handle the mentioned requirements;

5. Evaluation takes the implemented platform and benchmarks its features, in order to evaluate how its architecture handles realistic scenarios and how it compares with other potential architectures, as well as how its interface adheres to interaction design standards;

6. Conclusion and future work gives an account of the observed behaviors and caveats in the prototype during development and evaluation phases, relates the ways in which the proposed model for time series annotation improves on existing tools, and leaves a few clues to how the proposed platform can be iterated on in order to extend its capabilities and improve its overall performance and quality.

Publication

I submitted this study as a dissertation (equivalent to a thesis in the US) for the Master’s degree in Software Engineering at the University of Aveiro, passing with distinction with a grade of 19 out of 20, which in the US grading system corresponds to A (with A+ being the highest and F being lowest). The full text can be read on the institutional repository of the University of Aveiro.

To cite this research, you may use the following BibTex record:

@masterthesis{EdDuarte/master-thesis2018,
  author = {Duarte, Eduardo Miguel Oliveira},
  title = {Collaborative analysis over massive time series data sets},
  year = {2018}
}

Acknowledgements

The present study was developed in the scope of the Smart Green Homes Project [POCI-01-0247 FEDER-007678], a co-promotion between Bosch Termotecnologia S.A. and the University of Aveiro. It is financed by Portugal 2020, under the Competitiveness and Internationalization Operational Program, and by the European Regional Development Fund.

This work is licensed under a Creative Commons Attribution 4.0 International License.