Gaze Estimation and Prediction in the Wild

Appearance-based Gaze Estimation: What We Have Done and What We Should Do

Since its first appearance in the 90s, appearance-based gaze estimation has been gradually but steadily gaining attention until now. This talk aims at providing a brief overview of past research achievements in the area of appearance-based gaze estimation, mainly from the perspective of both personalization and generalization techniques. I will also discuss some remaining challenges towards the ultimate goal of camera-based versatile gaze estimation techniques.

Yusuke Sugano is an associate professor at Institute of Industrial Science, The University of Tokyo. His research interests focus on computer vision and human-computer interaction. He received his Ph.D. in information science and technology from the University of Tokyo in 2010. He was previously an associate professor at Graduate School of Information Science and Technology, Osaka University, a postdoctoral researcher at Max Planck Institute for Informatics, and a project research associate at Institute of Industrial Science, the University of Tokyo.

Measuring attention in interactions: from context based multimodal head pose analysis to 3D gaze estimation

Beyond words, non-verbal behaviors (NVB) are known to play important roles in face-to-face interactions. However, decoding non-verbal behaviors is a challenging problem that involves both extracting subtle physical NVB cues and mapping them to higher-level communication behaviors or social constructs. This is particularly the case of gaze, one of the most important non-verbal behaviors with functions related to communication and social signaling.

In this talk, I will present our past and current work towards the automatic analysis of attention (whether 3D gaze or its discrete version the Visual Focus of Attention, VFOA) in situations where large user mobility is expected and minimal intrusion is required. I will first introduce how we addressed VFOA recognition in meetings using Dynamical Bayesian Networks to jointly model speech conversation, gaze (represented by head pose), and task context. I will then present recent methods investigated to perform 3D gaze tracking, including robust and accurate 3D head pose tracking under 360 degrees as well as the use of several deep neural network architectures for appearance-based gaze estimation. The latter will include methods to build personalized models through few-shot learning and gaze redirection eye synthesis, differential gaze estimation, and online learning or adaptation, potentially taking advantage of priors on social interactions to obtain weak labels for model adaptation.

Dr. Jean-Marc Odobez received his PhD from Rennes University/INRIA in 1994 and was, from 1996 to 2001, Assistant Professor at the University of Maine, France. He is now a Senior Researcher at Idiap and adjunct faculty at the École Polytechnique Fédérale de Lausanne (EPFL) where he is a member of the School of Engineering (STI).

He is the author or coauthor of more than 150 papers, and has been the principal investigator of more than 14 European and Swiss projects. He holds several patents in computer vision, and is the cofounder of the companies Klewel SA and Eyeware SA companies. He is a member of the IEEE, and Associate Editor of the IEEE Transaction on Circuits and Systems for Video Technology and of Machine Vision and Application journals.

Jae-Joon Han is a Master of AI & SW Research Center at Samsung Advanced Institute of Technology (SAIT), the corporate research of Samsung Electronics. He received Ph.D degree in Electrical and computer engineering from Purdue University in 2006 and did a postdoctoral fellow at Purdue before he joined SAIT in 2007. Since then, he has mainly focused on developing computer vision and machine learning algorithms which enable for users to interact with devices in a novel way. He is currently leading a project related to on-device facial recognition. His research interest includes facial recognition, facial anti-spoofing, speaker verification, neural network model compression for on-device processing and gaze estimation.

Shalini De Mello is a Principal Research Scientist at NVIDIA. Her research interests are in computer vision and machine learning for human-computer interaction and smart interfaces. At NVIDIA, she has invented technologies for gaze estimation, and 2D and 3D head pose estimation, hand gesture recognition, face detection, video stabilization and GPU-optimized libraries for mobile computer vision. Her research over that past several years has pushed the envelope of HCI in cars and has led to the development of NVIDIA’s innovative DriveIX product for smart AI-based automotive interfaces for future generations of cars. She received doctoral and master’s degrees in Electrical and Computer Engineering from the University of Texas at Austin in 2008 and 2004, respectively.

Gaze into the future

Just a few years ago, eye tracking was still considered a research tool and a niche product used for e.g. offering people with assistive needs a voice. Today, however, eye tracking is broadcasted on major sports gaming tournaments and has become integrated into commercial off-the-shelf PCs and VR headsets.
A major pull for eye tracking technology today comes from the XR space. While eye tracking already now adds value to current XR products, future XR hardware, such as e.g. dynamic focus displays, simply won’t work without refined eye tracking solutions. In this talk we’ll describe what role eye tracking will play in future remote and near eye setups and where eye tracking simply needs to succeed.

Maria has been working with software and algorithm development for more than 15 years. During her career she has held a variety of roles from developer to research lead at companies like Philips, Infineon and Saab. Maria joined Tobii’s eye tracking algorithm team in 2015 as a Key Algorithm Engineer.

An eye tracker that works for every single individual is still a challenge. Maria has experienced from close how PC eye tracking has moved from being a specialty market and research tool to a mainstream gaming equipment. She has also been closely involved in the development of eye tracking for VR and AR, and been part of the work behind releasing products such as the HTC Vive Pro Eye.