Gaze represents an observable and reliable indicator of human attention: when we perform any activity, from driving a car to choosing a product on a supermarket shelf, knowing where our eyes point and how they move can convey crucial information about our cognitive processes and intentions. In other words, about what is going through our minds, and the decisions we are about to make. Researchers use eye tracking systems, in association with machine learning models, in various contexts, for example to detect changes in the state of attention, or changes in emotional state, or to help the user focus on relevant passages of a text.
Until a few years ago, accurate eye-tracking was only possible with sophisticated and expensive hardware and devices that were mainly used for laboratory experiments. Thanks to recent technological advances, these systems are becoming increasingly scalable and affordable. Today, there are eye-tracking systems that can accurately record eye movements on a large scale directly via the web, using front-facing cameras on common mobile devices such as smartphones and laptops, without any additional hardware or explicit calibration. WebGazer and SearchGazer are examples of applications developed for research purposes - but then also used by individuals and businesses - which use computer cameras to do eye-tracking of visitors to a website. It is therefore only a matter of time before ordinary users also start interacting with eye-tracking systems from their mobile devices on a daily basis, and this technology becomes a pervasive tool in everyday life, with the potential but also the risks that this entails.
Our paper "Predicting choice behavior in economic games using gaze data encoded as scanpath images", just published in Scientific Reports, is an example of the opportunities, but also of the risks that these systems entail. We used data collected during an experiment in which participants play against a computer opponent, a type of game commonly used in economics to study strategic interactions. Players know that they are faced with a computer that always makes the optimal and rational choice, the one oriented towards profit, and the player is also supposed to do the same.
The eye-tracking data recorded during this experiment were collected with highly accurate hardware and then analysed using scanpaths, i.e. coloured maps that allow the entire spatio-temporal sequence of eye movements to be traced. Using scanpaths, we demonstrate that some deep learning algorithms are able to accurately identify participants' moves before they make a decision. By showing the algorithm only a portion of the total eye movements made by the player to make his choice, we were able to correctly predict (and thus anticipate) the choice selected by the player and the type of strategy among the three options available in the game, outperforming more traditional methods capable of performing this type of analysis. For example, eye movements in the first 15 seconds of a game are sufficient for the algorithm to predict whether the player will choose the equilibrium strategy, providing a correct prediction 77 times out of 100.
The experiment shows us that, with the help of artificial intelligence, eye movements recorded in real time could be applied to anticipate a user's choices in a wide range of strategic contexts. This would benefit consumers, for instance by providing real-time feedback to avoid making wrong choices in strategic online contexts. But, perhaps even more plausibly, it could create a dangerous information asymmetry in favour of the automated system and those who control the technology.
In principle, in the specific context, an algorithm might be able to use this real-time information to anticipate a player's future choice. Furthermore, having included games characterised by different strategic structures in the study, we can show that our approach could detect an individual's choice even in strategic contexts never seen before by the algorithm.
Our results therefore suggest that eye-tracking data has the potential to favour those with the economic means to collect and process data with AI. Considering trends such as the growing popularity of virtual reality systems, the increasing participation in the gig economy via smartphone apps, and the development of devices that record eyes movement with high accuracy in a variety of conditions via smartphones and laptops, eye-tracking technology could become a beneficial tool for consumers. However, in order for it not to turn into yet another tool for the benefit of organized interests, public authorities and policy makers should supervise and efficiently regulate this new technology.
