Talking sight vs. sound with Chia-jung Tsay

Do classical-music listeners make judgments based on what they hear, or what they see? Are they really listeners, or are they viewers? Few are as qualified to answer the question as Chia-jung Tsay. With advanced degrees in organizational behavior and psychology (Harvard) and music (Peabody, Juilliard), Tsay, 32, is a visiting management professor at the Wharton School of the University of Pennsylvania from University College London who has studied more than a thousand casual concertgoers as well as musical experts. Tsay has come up with some startling challenges to the widely held assumption that sound dominates the judging process.

Q. Why did you decide to investigate the question of whether people rely more on sight than sound in evaluating performances?

A. As a classical pianist, for the last two decades, I've been performing and participating in competitions. It was through this experience that I realized, depending on the type of evaluation process involved - whether competitions required musicians to submit sound or video recordings - there could be very different results. This led me to my empirical investigations of whether people rely more on sight or sound in evaluating performance.

What kinds of visuals are people seeing that lead them to their conclusions? Are we talking about facial expressions, or, for a conductor, things like jumping and diving?

People are likely to have been influenced by characteristics such as passion, involvement, creativity, or uniqueness. Interestingly, although the characteristics are visually quite accessible, they appear to be less accessible through sound . . . . Visual information may be more powerful through its associations with expressive behavior and through its emotional impact. The physicality that produces great sound is the same physicality that can be visually evocative. So we likely associate passion, involvement, with quality of performance - and we infer the quality of performance from these characteristics. These can include facial expressions, body gestures, and levels of synchrony or group coordination.

How are these findings important to the business world, and what do they mean for musicians who spend most of their training time around sound production, not visuals?

These findings stem from the domain of music, which was a unique opportunity to study the impact of visual information since there's such consensus that it is a domain in which sound is much more informative.

So these findings hold wide implications for any type of professional judgment, any decision, any domain in which visuals are present but perhaps other information is more telling and more valued. Whether it involves hiring employees or selecting political leaders, we must be more mindful of our inclination to depend on visual information at the expense of the content that we actually value as more relevant for our decisions.

I believe there are elements of music performance that are channeled more convincingly through the visual [but] which also contribute to the quality of the sound of music - for example, gestures can reflect the physicality that produces certain touches, tones, and colors that underlie expressive sound.

Some teachers at conservatories seem to be quite attuned to the important role that visual information plays in the judgment of music, and they help their students become aware of its impact for effective performance. However, it is possible - and critics and professional musicians have noted this - that some performances can be "trained" or "choreographed" in a way that may no longer be authentic or true to the meaning of the composition, but may still remain effective as judged by audiences.

Do you think believing eyes over ears is the way we have always been wired, or is it a reflection of living in a more visual age?

The question is intriguing, and in fact is exactly the focus of several ongoing projects.