Pal-SPOT Project

Human activity recognition

Pal-SPOT* Project

1. Project definition

1.1 Background

Mobile and pervasive computing scenarios are characterized by the heterogeneity of mobile devices, network infrastructures, and usage conditions. Hence, automatically recognizing the situation under which a user is accessing a given service is of key importance for adapting that service to the user’s needs, requirements, and expectations. This theme is a hot research topic in the field of context-awareness; existing techniques can be roughly classified in ones based on symbolic reasoning and ones based on statistical reasoning.

1.2 Objectives

The aim of this project is to investigate the use of statistical reasoning for deriving high-level situations (e.g., the current activity of the user) from raw sensor data that characterize basic user’s parameters. In particular, the ultimate goal is to devise techniques for recognizing physical activities on the basis of data gathered from a restricted set of sensors that will be eventually integrated in many commercial mobile phones (e.g., accelerometer, GPS receiver, environmental sensors).

The expected outcome of this work is the integration of statistical reasoning with an existing middleware for context-awareness based on symbolic reasoning (CARE middleware [CARE]). A demonstrator application (called Pal-SPOT) will be released, that will be in charge of collecting raw data from Sun SPOT sensors and other sensors that a user brings in her pocket, automatically deriving situations such as the user’s physical and social activity, and interacting with an existing service to proactively notify the user about her closeness to interesting points-of-interest.

1.3 Use cases

UC1: Alice is travelling abroad for business, carrying her Pal-SPOT in her bag. At noon she leaves the train station towards the center of the city, where she’s expected for a business meeting at 1 pm. She has only a few minutes for having lunch before the meeting begins, so she walks at a quick pace. Her Pal-SPOT (on the basis of accelerometer data) recognizes that she’s walking fast; hence, it infers that Alice is in a hurry. Moreover, since it is noon, the Pal-SPOT derives that she is probably interested in finding a restaurant. Hence, on the basis of location data gathered from the GPS receiver, Pal-SPOT retrieves a list of nearby fast-foods from the remote point-of-interest service, and notifies Alice about their presence through an alert on her smart phone.

UC2: Bob trains in the park according to a personalized training plan, carrying a Pal-SPOT in his pocket. On the basis of sensor data, Pal-SPOT recognizes Bob’s speed and activity (jogging), and transmits the data to a virtual trainer integrated in his smart phone that is in charge of calculating the caloric consumption and suggesting exercises. On the basis of those data, the virtu al trainer suggests starting an exercise session in order to reach the expected caloric consumption amount. Hence, Bob asks the remote point-of-interest service for the nearest fitness trail, and obtains directions for that spot.

2. Preliminary architecture

The image below shows the overall system architecture.

In this version, due to technical problems in directly connecting SPOTs and PDAs, the information flow between sensors and the end-user device is mediated by a portable device equipped with a USB port (in our case, a Sony Vaio UX Series). The portable device is also in charge of providing Internet connectivity to the end-user device. However, in an ideal setting, sensor data are directly acquired by the end-user device, that is also in charge of sensor fusion, reasoning, and direct communication with the service provider.

3. Datasets

Using the proposed system we have collected 5-hours activity data (10 activities) for the experimental evaluation of our activity recognition techniques. The dataset has been used in [2], [3] and [5]. It can be downloaded here.