By Harish Katti and Mohan Kankanhalli
National University of Singapore, Singapore
harish2006@gmail.com
mohan@comp.nus.edu.sg
SUMMARY
This tutorial introduces eye-tracking as an exciting, non-intrusive method of capturing user attention during interaction with digital images and videos. The tutorial will focus on the following aspects,
1. What is eye-tracking and what does it offer for multimedia researchers?
2. Basic understanding of human anatomy, low and high level visual cognition, mathematical techniques underlying popular eye-tracking hardware and software.
3. Introduction to experiment design, analysis and visualization and application scenarios.
The tutorial will consist of two parts of roughly equal duration. The first half will introduce eye-tracking methodology and potential use in experiments and applications involving human interaction with images and videos. Second half will focus on hands-on experiments and data-analysis using state-of-art eye tracking hardware and software.
MOTIVATION AND BACKGROUND
This tutorial introduces eye-tracking as an exciting, non-intrusive method of capturing user attention during interaction with digital images and videos. Our motivation stems from (a) huge volumes of image and video content, generated as a result of human experiences and interaction with the environment. We frequently encounter video and image collections having millions video clips on YouTube and billions of images on repositories such as Flickr or Picasa. It becomes useful and necessary to automate the process of understanding and storing it to enable subsequent use such as indexing, retrieval and query processing, re-purposing for devices with different form factors. (b) Personalization and interaction in human-media is an important and promising direction of research. Having access to individual preferences and behavioral patterns is a key component of such a system. Eye-gaze can play a valuable role in this context. Recent developments in hardware technology have made it possible to seamlessly integrate eye-tracking into laptops and mobile devices. This opens up a plethora of possibilities for research and applications for applications and frameworks that can respond to user’s visual attention strategies. (c) Visual content design such as in advertising often employs techniques that guide user attention to produce visual impact and elements of surprise and emotion. Eye-gaze has been used as a tool to evaluate different choices of visual elements and their placement. (d) Affective analysis of images and videos is an ongoing and challenging area in multimedia research. We show how eye-gaze and accompanying pupillary dilation information can aid affective analysis. Recent efforts on use of pupillary dilation for affective analysis for videos will be presented.
LIST OF TOPICS
The first half will cover the following topics,
1. Defining the role of eye-gaze for an experimental study or application and preparation of stimulus set for image or video based experiment.
2. Eye-tracking hardware choices ranging from low-cost open source, desktop based and mobile head-mounted options.
3. Typical eye-gaze analysis and visualization methods for image and video based experiments.
4. Interactive, eye-gaze contingent systems.
The second half will consist of a hands-on session covering the following topics,
1. Aspects of analysis and visualization for image and video stimuli. This will be introduced with customized code accompanying NUSEF dataset, as well as commercial software from SMI.
2. Eye-tracking experiments with image and video stimuli using commercial trackers from SMI and also low-cost open source based tracking.
INTENDED AUDIENCE
The tutorial is aimed at researchers who want to understand the basics of eye-tracking and potential applications to image and video. Participants can be from diverse backgrounds like psychology, engineering and computer science and will be exposed to methods that are complementary to their basic training. A basic knowledge of programming and statistics will help, but is not a compulsory pre-requisite as the tutorial material will be self-contained and introductory in nature. We can accommodate 15-20 participants due to hardware constraints. SMI has kindly agreed to sponsor the commercial eye-tracking hardware and up to 5 merit based scholarships of 100 dollars each.
TENTATIVE SCHEDULE
The tutorial will be spread over two halves of about 1.5 hours. The first half will consist of a series of half hour lectures addressing the following questions,
1. What is eye-tracking and what does it offer for multimedia researchers?
2. Basic understanding of human anatomy, low and high level visual cognition, mathematical techniques underlying popular eye-tracking hardware and software.
3. Introduction to experiment design, analysis and visualization and application scenarios.
The second half will consist of hands-on sessions where the participants will be split up into groups of 3-4 people. Each group will be assigned to a hardware setup and expected to conduct a small eye-tracking study or visualization exercise. The groups will be guided by the speakers through the exercises and will help the participants gain experience of practical eye-tracking experiments.
RELATED MATERIAL
We will provide background reading and introductory material to selected participants well in advance. The participants will be provided with additional reading material during the tutorial and this will include how-to’s and FAQ’s for the hands-on sessions. We also intend to provide an overview and survey over different free and commercial eye-tracking setups, datasets and related software to help interested participants start on eye-tracking research.
ORGANISERS AND BACKGROUNDS
Harish Katti received the B.Engg. degree in Computer science and Engineering from Karnatak University and M.Tech degree in Bio-Medical Engineering from IIT Bombay. He is currently a PhD candidate in the Department of Computer Science, School of Computing, National University of Singapore. He has worked the area of multimedia systems in Sasken Communications Pvt Ltd and Emuzed India Pvt Ltd. His current research interests are in visual perception and applications of eye-tracking methodology to media applications.
Mohan Kankanhalli obtained his BTech (Electrical Engineering) from the Indian Institute of Technology, Kharagpur and his MS/PhD (Computer and Systems Engineering) from the Rensselaer Polytechnic Institute. He is a Professor at the School of Computing at the National University of Singapore. He is on the editorial boards of several journals including the ACM Transactions on Multimedia Computing, Communications, and Applications, Multimedia Systems Journal and Multimedia Tools and Applications. His current research interests are in Multimedia Systems (content processing, retrieval) and Multimedia Security (surveillance, privacy and digital rights management).