At Google in Mountain View I worked on modelling and prediction of user performance, using the tools of machine learning / Tensorflow. The work also includes an eye-tracking study that provides an in-depth analysis of visual attention on mobile phones. It was presented April at the CHI conference in Montreal, Canada.
The work is essentially trying to model every little subtask that is involved in interacting with a grid interface. Scrollable grid interfaces are common on mobile phones, be it the gallery, app homescreen, or any other. Understanding user performance is key to improving the usability of such UIs. With our model, we can predict how much time it takes to scroll & select an item in a grid, further we uncover many performance characteristics that are involved, ranging from visual search (using eye-tracking), manual scrolling (touch gestures), and tapping to select an item (Fitts Law).
User wearing a mobile eye tracker while interacting with a smartphone
One of the most interesting findings is that users have two strategies — they can either start scrolling down from the top (top-down strategy), or immediately do a “hard” swipe to the bottom of the UI, and then scroll up (bottom-up strategy). How to model such contrasting strategies? We found that for our tests, 20% of the sessions involved the bottom-up strategy, and it was strongly affected by the name of the target. I.e., when searching for “Twitter” in the app-list, the app icon is naturally closer to the bottom of the list (as it is sorted alphabetically). The user knows that, and thus swipes down to improve the search.
However, from the bottom, when scrolling up steadily, users showed performance that is linear depending on the row that the target is located at. The same for the top-down strategy, only that it is starting at the top. Thus, the performance of the strategies is equal, with the difference being that initial swipe down gesture for the bottom-up strategy. In sum:
Top down strategy = a + row * b
Bottom up strategy = swipe_down + a + (maxRows – row) * b
There are much more performance factors involved in the user interaction, so if you are still reading until here, I refer to the paper:
Analysis and Modelling of Grid Performance on Touchscreen Mobile Devices
Ken Pfeuffer, Yang Li. 2018. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’18). ACM, Montreal, QC, Canada. doi, pdf
Ken Pfeuffer 