Blog

  • History of Eyes and Hands for Computer Control

    History of Eyes and Hands for Computer Control

    Apple’s Vision Pro provides the first operating system fully designed for our eyes and hands. Perhaps the first-ever consumer device. A game-changer for XR like the first IPhone’s “multi-touch” effect that transformed the industry. In 1644, René Descartes, a seminal figure in the emergence of science, depicted the eye-hand coordination as a fundamental bridge between

    Read more

  • Semi-automatic 3D UI Placements

    Semi-automatic 3D UI Placements

    There’s a line of research in human-computer interaction that is concerned with the question: where should be ideally place a UI in 3D environments? Close to the line of sight for high visibility? Close to the hand, to reach it immediately? Or off to the side, to avoid occluding the center view? Many potential goals

    Read more

  • [SUI’23] PalmGazer: One-handed interaction in AR using eye and hand tracking

    [SUI’23] PalmGazer: One-handed interaction in AR using eye and hand tracking

    This paper has been published at the ACM Symposium on Spatial User Interaction 2023 in Sydney, Australia, and received a Best Paper Honorable Mention Award! Background One-handed interaction has become a ubiquitous aspect of our daily lives, primarily through the use of smartphones. However, as augmented reality (AR) smartglasses and mobile head-mounted displays (HMDs) become increasingly

    Read more

  • [CHI’23] Fitts Law Study on 3D Gaze and Gesture Selection Techniques

    [CHI’23] Fitts Law Study on 3D Gaze and Gesture Selection Techniques

    3D virtual and real environments can be vast, and the interaction with targets near and far is not easy. You could have a controller for raypointing, but this requires users to always carry the device. Freehand gestures have been long considered as an alternative, natural UI. However, direct manipulation becomes difficult over distance. But how

    Read more

  • [ETRA’22] Gaze-hand Alignment based text entry

    [ETRA’22] Gaze-hand Alignment based text entry

    Text entry is yet one of the most challenging tasks in 3D environments (VR, AR, XR). Probably the most robust is currently to use the free hands and the index finger to directly type into a floating QWERTY keyboard. This is currently used by most XR devices (e.g., Microsoft Holo Lens 2). However, moving the

    Read more

  • [ETRA’22] Gaze-Hand Alignment: Gaze and Hand Pointers for Interacting with Menus in Augmented Reality

    [ETRA’22] Gaze-Hand Alignment: Gaze and Hand Pointers for Interacting with Menus in Augmented Reality

    Gaze and gestures are well suited to complement Augmented Reality (AR) and interactions that are situated in the world. Current trends in AR technology reflect this with the integration of both hand- and eye-tracking in headworn display (HMD) devices. However, the design of input techniques that rely on gaze and freehand gestures is challenging, as

    Read more

  • [CHI’21] SpatialProto: Rapid Prototyping of Interactive Mixed Reality

    [CHI’21] SpatialProto: Rapid Prototyping of Interactive Mixed Reality

    Prototyping in Virtual Reality is quite difficult – and more so in Augmented and Mixed Reality (MR) where the physical and virtual realities unify. In this project, we propose a novel approach and system to create Mixed Reality experiences rapidly, for anyone who has a typical MR headset. Called SpatialProto, the system allows users to

    Read more

  • [Computer & Graphics ’21] ARtention: a design space for gaze-adaptive user interfaces in augmented reality

    [Computer & Graphics ’21] ARtention: a design space for gaze-adaptive user interfaces in augmented reality

    Ever wondered that with AR glasses, you might simply look outside your window into the nature, and augmented information only appears when we really want to? Like, I am curiously gazing at that particular mountain far-away, and subtly the name and height of it appears. This question underlies this research, focusing on how our eyes

    Read more

  • [VRST’20] Gaze-enhanced Menus in Virtual Reality

    [VRST’20] Gaze-enhanced Menus in Virtual Reality

    We look at the menu before we interact with it – which is what we aim to exploit in this project via eye-tracking. In VR 3D design tools, users often hold a menu in one hand, while the other hand is reserved for the drawings with a pen. To change something in a menu, we

    Read more

  • [CHI’19] Adaptive VR by Implicit Behavioural Biometrics

    [CHI’19] Adaptive VR by Implicit Behavioural Biometrics

    We explored a mechanism to authenticate users in VR without effort in the user’s background. This can become quite important, considering the spread of head mounted devices in AR, VR, MR or XR, and the never-ending effort of having to authenticate to every session and use of a device. Of course, the headset itself could

    Read more