Humans and Recommender Systems: Towards Mutual Understanding

Recommender systems (RS) are a central means for supporting users in dealing with the information overload problem (e.g., in online shopping or on streaming platforms). Mostly, RS rely on some form of collaborative filtering, where recommendations are computed based on neighboring users or items. These approaches, however, neglect two important elements when modeling users and RS, leading to a mutual misunderstanding: firstly, RS are not able to capture the actual human decision-making that leads to choosing certain items, and secondly, RS are hardly able to communicate the rationale behind recommendations. In this project, we focus on music recommendations and strive to enhance the understanding of human decision-making underlying the choice of music in a given situational context. Moreover, we aim to advance the users` understanding of the decisions that lead to the recommendation of certain (sequences of) tracks. We believe that an increased understanding and communication between users and the system can contribute to improved user models and, thus, recommendation performance. A previously largely unexplored aspect will be the development of techniques for sequential recommendation strongly targeted at explanations and considering user feedback. Our research goals are as follows: understanding and modeling user intent by approaching the task from two different perspectives: (i) gaining a detailed understanding of user intent on the individual level through interviews and (ii.) understanding intent by exploiting large-scale data of millions of listening histories, devising models for explanation of sequential recommendations and incorporating feedback in multi-faceted feature spaces, including dimensions of music content, listener intent, and listening context, and researching consistency of theories (e.g., influence of personality on music listening behavior) and our models created in a data-driven manner from large-scale user-generated data, and using respective findings to enhance our user and RS models. We will adopt data and hypothesis-driven research methods; findings from both perspectives will be connected to existing theories and used to refine the models of user intent and explanations of recommendations. The developed models and techniques are evaluated by quantitative (also including beyond-accuracy measures) and qualitative means (e.g., structured interviews or task-driven user observations). The project consortium is composed of five Austrian researchers with complementary expertise: Eva Zangerle (University of Innsbruck, Department of Computer Science), Markus Schedl (Johannes Kepler University Linz), Peter Knees (Vienna University of Technology), Marcel Zentner (University of Innsbruck, Department of Psychology) and Michael Huber (University of Music and Performing Arts Vienna).

Recommender systems (RS) play a central role in helping users navigate the overwhelming abundance of digital content, such as in online shopping environments or on music and video streaming platforms. Traditionally, these systems are built upon collaborative filtering techniques, where recommendations are derived from similarities between users and/or items. However, such approaches overlook two critical aspects: (1) the complex human decision-making processes that drive the selection of content, and (2) the inability of RS to explain the reasoning behind their recommendations, leading to a mutual lack of understanding between users and systems. This project focuses on music recommendation and aims to deepen our understanding of the human decision-making involved in music choice within specific situational contexts. In music consumption, sequential context plays a particularly important role, as listeners typically experience music as a temporal sequence of listening events. One key pillar of the project was the development of novel sequential recommendation algorithms that can effectively model and utilize temporal listening behavior. A second major focus of the project was on the emotional context in which music is consumed. To recommend music that aligns with a user's emotional state, it is essential to understand the emotional impact of music. To this end, we created a high-quality database-EMMA (Emotion-to-Music Mapping Atlas)-that captures the emotional effects of hundreds of music excerpts across diverse genres. These effects were rated by humans using the Geneva Emotion Music Scale (GEMS), a tool specifically designed to reflect the nuanced emotional experiences evoked by music. Building on this data, we developed an autotagging system trained on EMMA that can automatically annotate new music tracks with GEMS-based emotional tags. This facilitates the development of emotion-aware music recommender systems at scale. The third pillar of the project addresses two interrelated goals: integrating (negative) user feedback into the recommendation loop and providing users with interpretable explanations for recommendations. This approach is intended to enhance both user trust and satisfaction, ultimately leading to more transparent and user-aligned music recommendation systems. To complement these technical developments, we also conducted structured interviews to gain a deeper understanding of how people perceive and interact with music recommender systems in their daily lives. The project consortium was composed of five Austrian researchers with complementary expertise: Eva Zangerle (University of Innsbruck, Department of Computer Science), Markus Schedl (Johannes Kepler University Linz), Peter Knees (Vienna University of Technology), Marcel Zentner (University of Innsbruck, Department of Psychology) and Michael Huber (University of Music and Performing Arts Vienna).