You are here

Rethinking the Test Collection Methodology for Personal Self-tracking Data

Authors: 

Frank Hopfgartner, Cathal Gurrin, Hideo Joho

Publication Type: 
Refereed Conference Meeting Proceeding
Abstract: 
While vast volumes of personal data are being gathered daily by individuals, the MMM community has not really been tackling the challenge of developing novel retrieval algorithms for this data, due to the challenges of getting access to the data in the first place. While initial efforts have taken place on a small scale, it is our conjecture that a new evaluation paradigm is required in order to make progress in analysing, modeling and retrieving from personal data archives. In this position paper, we propose a new model of Evaluation-as-a-Service that re-imagines the test collection methodology for personal multimedia data in order to address the many challenges of releasing test collections of personal multimedia data.
Conference Name: 
MultiMedia Modeling. MMM 2020
Proceedings: 
Proceedings of MultiMedia Modeling. MMM 2020.
Digital Object Identifer (DOI): 
10.1007/978-3-030-37734-2_38
Publication Date: 
01/05/2020
Conference Location: 
Greece
Research Group: 
Institution: 
Dublin City University (DCU)
Open access repository: 
Yes