Following The Finger

New eye tracking technologies offer the possibility to study with great precision the gaze movements of people while they are looking at paintings. Among the different factors presumed to guide the attention of art beholders there is one very obvious element that has never been studied in-depth, neither theoretically nor empirically: the pointing gestures. It is a well-known fact that, during the Early Modern period, painters depicted multiple pointed forefingers in all kind of pictorial topics and genres, in order to facilitate the understanding of the content. Usually, the pointing chirograms (a graphically represented hand sign) have major roles: highlighting key elements of the story, revealing important connections between the characters, showing directions to take, indicating less visible details, etc. Some compositions are simple with only one pointed finger while others contain multiples gestures, pointing in different directions and carrying different meanings. In this project we will make for the first time a large statistical survey of pointing gestures in Early Modern painting and establish relevant typological categories according to their functions. This will allow us to know exactly how often do painters use these signs and why. Are pointing gestures in painting similar to pointing gestures in real life or in Sign Language? Or do they have some unique linguistic properties? In a second step we will proceed to the eye tracking experiments, by selecting some of the most interesting examples of paintings with pointing chirograms and showing them to different groups of beholders. We want to know not only whether and how the pointing fingers will influence the art perception but also whether the personal background of the beholders will interfere somehow. This is why we will constitute three different groups of participants: 1) art experts; 2) lay people and 3) Deaf native speakers of Austrian Sign Language. Each one of these three groups could have a particular sensibility to the pictorial gestures. Are pointed forefingers universally understood or does their perception depend on the knowledge and culture of the viewer? The pictorial gestures constitute a remarkable but understudied language. By researching the understanding of this language we could significantly enlarge our knowledge about art perception, both in cognitive and cultural terms.

In this project, we used eye-tracking technology and open interviews to investigate two main research questions: (1) how depicted pointing fingers influence the perception of early modern paintings, and (2) how Deaf people who are fluent in Austrian Sign Language (ÖGS) perceive such paintings. Pointing gestures are frequently used by painters to direct viewers' attention toward important elements within a painting. Art historical theories and ancient sources have long suggested that gestures play a key role in structuring visual narratives, and pointing fingers are among the most commonly represented gestures in Western art. However, it has rarely been empirically tested whether viewers actually follow these gestures. We recorded the eye movements of 84 participants while they viewed 15 narrative paintings from the 16th and 17th centuries containing pointing gestures. Each participant saw either the original version of a painting or an edited version in which the pointing gestures had been removed, allowing us to compare perception with and without gestural cues. The results show that depicted pointing fingers significantly influence visual attention in several ways. First, they strongly increase attention to the face of the pointing figure, suggesting that viewers seek to understand the intention and emotional state of the person performing the gesture. Second, pointing gestures also enhance attention to the indicated target areas. Finally, we found that pointing gestures affect how viewers relate different figures within a composition, shaping the overall reading of the scene. Although the pointing hands are only briefly fixated, they have a strong effect on the global organization of visual attention. The second research question focused on how Deaf signers perceive early modern paintings. We compared three groups: 30 Deaf signers, 30 hearing non-experts, and 28 art history students (expert group). The results show that Deaf signers and art experts display remarkably similar viewing patterns. Both groups explore images more broadly, show shorter fixations, and pay greater attention to gestures and symbolic objects. In addition, in interviews both groups used more concrete language when describing the paintings. While these similarities are striking, they arise from different sources. The viewing patterns of art experts are shaped by academic training, whereas Deaf signers bring different visual and linguistic experiences that influence how they explore complex images. Our results suggest that these experiences may foster a more dynamic and exploratory way of looking, even in the absence of formal art historical training. These findings do not imply that Deaf signers interpret paintings in the same way as experts, but they suggest that they may approach visual heritage in an attentive and engaged manner. Overall, the study highlights how both visual language mastery and sensory experience associated with deafness can shape the way viewers engage with artworks.