Towards an experimental narratology of the image

The question of the temporal nature of the arts has occupied researchers since Lessing affirmed that an image differs from a text because it does not have any temporal program. Until now, some image theorists and narratologists believe this to implicate that a picture cannot tell a story, while many art-historians believe it to be quite unproblematic that many pictures tell stories. They are thus more interested in the how. The aim of the project is to use psychophysical methods in order to experimentally extend on these debates and especially to shed light on the relationship between visual exploration of a picture in time and its degree and type of narrativity. In his doctoral thesis, the applicant Speidel (Paris Sorbonne, philosophy) theoretically analyzed the possibility of creating purely visual narratives in a single still image. He now wants to empirically evaluate the possibilities and the modalities of such narratives. The co-applicant, Rosenberg (University of Vienna, art-history), established the first eye-tracking-lab in an art-history department and developed tools to measure and analyze gaze-behavior while looking at paintings. He discovered that even though subjects do not perceive all picture elements in the same order, their gazes very often repeat specific paths. A pilot-study based on Speidel`s hypotheses and performed in Vienna demonstrated that there is some inter-subjective stability concerning the order in which specific picture elements are perceived (some tend to be seen earlier, some later). The heart of the project is a study in which subjects will look at different kinds of pictures (highly narrative or not) while their psychophysical reactions are being tracked through various means. We will use 1. eye-tracking to analyze which picture zones are perceived earlier, which later; 2. facial EMG and Skin Conductance Response (SCR) to understand the emotional reactions evoked when viewing different picture elements; and 3. verbal descriptions to understand whether and how paintings are perceived as narratives. We hypothesize that narrative structures, as soon as they are recognized, significantly affect visual exploration patterns. We further believe that certain pictures have by their makers been intended to evoke emotions such as suspense, surprise and relief, which have been frequently associated with narrative. We hope that psychophysical techniques will allow to correlate such emotions with certain picture elements. For the applicant, who has been formed in philosophy and aesthetics, joining the team in Vienna will be a valuable interdisciplinary experience opening his way into cognitive science and experimental image theory. For the laboratory in Vienna, working with Speidel opens up a new field of research, which is expected to have considerable impact on art-history. By clarifying the differences and similarities of picture- and text-reception, the project will be highly relevant for text-and-image-studies.

A monochronic picture that only shows one moment in time can autonomously convey a full new story to different audiences. In 1766, G. E. Lessing asserted that monochronic paintings are not suitable for narrative, because a picture that only shows one moment can not convey events that do not take place simultaneously. To this day, many image theorists and narratologists believe that such pictures can only remind viewers of stories which they know already. My project challenges these ideas theoretically and empirically. Based on examples such as Théodore Géricault`s Raft of Medusa, I argue that single monochronic pictures can convey all elements that are necessary to tell a minimal story. When seeing people on a self-made raft, we understand that shipwreck preceded and a ship appearing on the horizon lets one conceive the possibility that they will be rescued. Certain elements e.g. the shipwreck, or a ship and potential rescue on the horizon might even evoke suspense and relief in the viewer, responses which are characteristic of the reception of narrative developmenst. But if they do, then narrative qualities are not limited to vehicles of narration like verbal narratives, films and series of pictures. These reflections have been put to the test in experiments with more than 200 non- expert viewers, who largely agreed that certain monochronic pictures tell stories. The difference with pictures with lower narrativity was statistically significant. As storytelling is a concept that non-experts master well, this result calls into question the conviction that monochronic pictures cannot tell stories. Further analyses focused on how strongly content summaries of different participants resembled each other and whether different viewers reconstructed the same timelines. It appeared that a relatively low number of scripts are used when different participants sum up the content of a picture. To take an example: in the Bagnio Scene from William Hogarths Marriage-à-la-Mode shows a woman is kneeling in front of a man who has been wounded, while someone flees through a window. The viewers either spoke of a duel or a murder. To sum up the content of Banksys Media, which shows a young girl in a landscape with destroyed buildings, only the scripts natural catastrophe and destruction by war were used. Participants regularly mention actions that are not explicitly represented. It appeared that such mentions were based on depicted traces: Fallen chairs in a room and blood on a mans shirt, to take an example, lead participants to talk about a fight, even when no conflict was explicitly depicted. In this context, it should be noticed that different viewers reconstructed the temporal order of events shown and implicated in different pictures in very similar ways. In a work like Hogarths Bagnio Scene more than 50% considered the same event the start or the end of the chain of events that is depicted. It thus seems uncontestable that pictures convey events that they do not explicitly depict and that pictures can autonomously convey stories by doing so.