Lifestyle Understanding through the Analysis of Egocentric Photo-streams

Autor/a

Talavera Martínez, Estefanía

Director/a

Radeva, Petia

Petkov, Nicolai

Fecha de defensa

2020-02-14

Páginas

174 p.



Departamento/Instituto

Universitat de Barcelona. Departament de Matemàtiques i Informàtica

Resumen

Describing people’s lives has become a hot topic in several disciplines. Lifelogging appeared in the 1960s as the process of recording and tracking personal activity data generated by the daily behaviour of a person. The development of new wearable technologies allows to auto- matically record data from our daily living. Wearable devices are light-ware and affordable, which shows potential for the increase of their use by our society. Egocentric images are recorded by wearable cameras and show a first-person view of the life of the camera wearer. These collected images show an objective view of the daily life of a person and thus are a rich source of information about her or his habits. However, there is lack of tools for the analysis of collections of egocentric photo-sequences and thus room for progress. This thesis investigates the development of automatic tools for the analysis of egocentric images with the ultimate goal of getting understanding of the lifestyle of the camera wearer. This work addresses five main topics in the field of egocentric vision: 1. Temporal photo-sequences segmentation: We introduce an automatic model for the defi- nition of temporal boundaries for the division of egocentric photo-sequences into mo- ments, which are sequences of images describing the same environment. The model is based on global and semantic features and achieves a 66% F-score over the EDUB-Seg dataset. 2. Routine discovery: We propose an automatic tool for the discovery of routine-related days and the visualization of patterns of behaviour, based on the use of topic modelling over semantic concepts extracted from the photo-sequences. The introduction of the EgoRoutine dataset composed of a total of 104 days is part of this work. The model is able to classify days into routine and non-routine related with an accuracy of 80%. 3. Food-related scenes recognition: We introduce a hierarchical classifier for the recognition of visually highly similar food-related images into 15 different classes that describe daily activities related to food consumption, acquisition, and preparation. We intro- duce the EgoFoodScenes dataset, which our model is able to classify into the 15 cate- gories with an accuracy of 68%. 4. Sentiment retrieval: We explore the sentiment associated with images by classifying them into Positive, Neutral, and Negative. Our model is based on the analysis of global features and obtained semantic concepts with associated sentiment. We obtain an ac- curacy of 75%. Results show that positive images relate to outdoor environments or with social interactions, neutral to work-related environments, and negative to non- informative or visually not clear images . 5. Social pattern characterization: We propose a model that characterizes the social be- haviour of the camera wearer based on the occurrence of people that the camera wearer meets throughout her/his data collection. The proposed social parameters allow the definition of a radar chart that shows its potential for the comparison of social patterns among individuals. The introduced and made publicly available egocentric datasets and the obtained results in the different performed experiments indicate that behaviour can be identified and studied. We conclude that the developed automatic algorithms for the analysis of egocentric images allow a better understanding of the lifestyle of the camera wearer. Applications based on the analysis of this data can lead to the improvement of the quality of life of people and therefore, are worth to continue exploring.

Palabras clave

Visió per ordinador; Visión por ordenador; Computer vision; Interès personal; Interés personal; Self-interest; "Lifelogging"

Materias

62 - Ingeniería. Tecnología

Área de conocimiento

Ciències Experimentals i Matemàtiques

Nota

Programa de Doctorat en Matemàtica i Informàtica / Tesi realitzada conjuntament amb la Universitat de Groningen

Documentos

ETM_PhD_THESIS.pdf

19.89Mb

 

Derechos

ADVERTIMENT. Tots els drets reservats. L'accés als continguts d'aquesta tesi doctoral i la seva utilització ha de respectar els drets de la persona autora. Pot ser utilitzada per a consulta o estudi personal, així com en activitats o materials d'investigació i docència en els termes establerts a l'art. 32 del Text Refós de la Llei de Propietat Intel·lectual (RDL 1/1996). Per altres utilitzacions es requereix l'autorització prèvia i expressa de la persona autora. En qualsevol cas, en la utilització dels seus continguts caldrà indicar de forma clara el nom i cognoms de la persona autora i el títol de la tesi doctoral. No s'autoritza la seva reproducció o altres formes d'explotació efectuades amb finalitats de lucre ni la seva comunicació pública des d'un lloc aliè al servei TDX. Tampoc s'autoritza la presentació del seu contingut en una finestra o marc aliè a TDX (framing). Aquesta reserva de drets afecta tant als continguts de la tesi com als seus resums i índexs.

Este ítem aparece en la(s) siguiente(s) colección(ones)