New Framework Enhances AR Experience by Predicting Where Users Will Look

Augmented reality (AR) devices like smart glasses may soon be able to predict where a user will look and provide an enhanced interactive experience.

Fiona Ryan, a Ph.D. student in Georgia Tech’s School of Interactive Computing, is pioneering research that tracks and predicts user gaze from a first-person perspective in 3D environments.

Currently, most AR devices react to where users look, playing catch-up. Ryan’s method could give these devices a heads-up and make the user experience more seamless.

“It allows an AR system to anticipate what the person will interact with next and where they’re going to look next so it can proactively render the experience,” she said.

Ryan is the lead author of the paper Forecasting 3D Scanpaths in Egocentric Video, which she will present next week at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) in Denver.

While there is existing research on predicting user gaze from 2D still images, her work is the first to address the issue through a 3D framework.