Objectives:

Home monitoring is an important topic of research in HCI. Some such applications are being developed for patient monitoring, elderly care, office monitoring, surveillance systems etc. One recent survey on home activity monitoring concludes that the major limitations of camera based recognition of human activity are:

low-light or no-light condition during moments of capturing image, and

unavailability of depth information.

The review also suggests that segmentation and tracking of multiple persons in video may be improved using depth image. Meanwhile Microsoft has launched Kinect as a gaming platform in 2011. Kinect has an RGB camera and a depth sensor. The popularity of Kinect, and as a result, availability of depth information has added a new dimensions of research in the field of HCI involving monitoring of home activity.

Activity recognition using Kinect can be broadly grouped into two classes:

those using RGB-Depth values, and

those using skeleton points (obtained directly from Kinect API).

While the RGB-Depth value of Kinect already has several applications related to home monitoring, the depth data is often unreliable due to limitations of the Kinect sensor. The skeleton data of a moving person is obtained as a stick model with 3D coordinates of 20 major joints of human body. In this tutorial we will present

pre-processing including de-noising issues of the Kinect data,

feature estimation and condensation techniques for the Kinect data, and finally

We will also indicate the possible outcome of fusion of both RGB-Depth data and skeleton representation and provide a comparison of existing approaches on benchmark datasets.

Content and Benefits:

The tutorial is planned to be of 3 hour duration with a 10 minute tea-break. The possible table of content for this presentation would be tentatively as follows:

Overview of Home Activity Recognition and Human Interaction Modeling prior to onset of Kinect: 10 min

Overview of Kinect: 10 min

Preprocessing: 20 min

Why it is necessary

What has been done so far

What should be done

RGB-D based methods: 40 min

Features Used

Methods Used

Merits and demerits of different methods and features

Comparison of different methods

Skeleton based methods: 40 min

Features Used

Methods Used

Merits and demerits of different methods and features

Comparison of different methods

Fusion of RGB-D and Skeleton based approaches: 20 min

Applications in HCI: 10 min

Related Datasets: 10 min

Related Conference/workshop and communities: 10 min

The tutorial aims to provide a complete understanding of the states-of-the-art in the field of automatic recognition of activities related to home monitoring. The use-cases in this field are expanding and posing a number of challenges for research. The tutorial is expected to provide a thorough knowledge of the use of Kinect device data, especially the depth values, and its limitations. The attendee should be able to appreciate an end-to-end solution pipeline of a home monitoring system. The presentations will also include a survey and a set of application-driven research scopes.

Target Audience:

Researchers, scholars and people from industry who are working on HCI technology using computer vision.

Bio Sketch of Presenters:

Prof. Dipti Prasad Mukherjee: He is a Professor at the Electronics and Communication Sciences Unit, Indian Statistical Institute, Kolkata, India. He has written more than 95 peer-reviewed research articles. His primary area of research is image processing and computer vision.

Dr. Tanushyam Chattopadhyay : He is working as a senior scientist in TCS Innovation Labs of Kolkata. He has worked in image and video processing. He has published nearly 50 papers in peer reviewed journals and conferences and also have plenty on Indian and international patents. He has recent publications on Kinect based technologies in Ubicomp and many are in the press. He is currently leading the Kinect based home monitoring project in the lab.

Mr. Kingshuk Chakravarty: He is working as a senior researcher in TCS Innovation Labs of Kolkata. He has worked on HCI technologies using image, video and signal processing. He has published nearly 6 papers in peer reviewed journals and conferences like Sensys, CEC, IEEE SMC etc and some are yet to be published. He is currently working on Kinect based HCI technology.