ScanGAN360: A Generative Model of Realistic Scanpaths for 360° Images

Note: We don’t have the ability to review paper

PubDate: February 2022

Teams: Universidad de Zaragoza;Stanford University

Writers: Daniel Martin; Ana Serrano; Alexander W. Bergman; Gordon Wetzstein; Belen Masia



Understanding and modeling the dynamics of human gaze behavior in 360° environments is crucial for creating, improving, and developing emerging virtual reality applications. However, recruiting human observers and acquiring enough data to analyze their behavior when exploring virtual environments requires complex hardware and software setups, and can be time-consuming. Being able to generate virtual observers can help overcome this limitation, and thus stands as an open problem in this medium. Particularly, generative adversarial approaches could alleviate this challenge by generating a large number of scanpaths that reproduce human behavior when observing new scenes, essentially mimicking virtual observers. However, existing methods for scanpath generation do not adequately predict realistic scanpaths for 360° images. We present ScanGAN360, a new generative adversarial approach to address this problem. We propose a novel loss function based on dynamic time warping and tailor our network to the specifics of 360° images. The quality of our generated scanpaths outperforms competing approaches by a large margin, and is almost on par with the human baseline. ScanGAN360 allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior, facilitating experimentation, and aiding novel applications in virtual reality and beyond.

You might be interested in …

【中国科学报】X射线一照 “炮弹”脱“糖衣”


放射治疗激活阿霉素前体药物的示意图。研究团队供图   手术、化疗、放疗是肿瘤、癌症等疾病的常见治疗手段。传统化 […]

Read More



  今年一季度,受原料成本压力,物流压力以及企业订单减少等因素影响,我国家纺行业利润总额同比下降23.63%。 […]

Read More



    近年来我们启盟律师楼一直会接到这样咨询,比如某华侨是在中国领事馆登记结婚的,结婚前女方有一套 […]

Read More


邮箱地址不会被公开。 必填项已用*标注

  • 友情链接