The Universal Sense (21 page)

It might seem odd to think of movies as applied neuroscience and psychology, but “neurocinema” is very trendy these days as a tool for examining how people’s brains react to films in order to improve their effectiveness. One approach that’s being
taken by a San Diego group called MindSign Neuromarketing is to have people watch a short section of film and then examine their brain blood flow in an fMRI, usually focusing on the amygdala after watching a horror film. This is a very sexy approach, but it has some important limitations. For one, it’s very expensive. For another, in order for researchers to gather data at many points during the movie, the subject has to watch very short chunks, which interrupts the flow of the film and therefore limits the value of the information. In addition, a single fMRI scan takes at least two seconds, during which you go through several hundred perceptual events. So the data are neurally fuzzy and unlikely to yield anything more accurate than what you’d get by watching a filmgoer’s face and movements during a movie.

The soundtrack in film or video is composed of both the film score (the underlying music that provides an emotional timeline for the work) and the dialogue and environmental sounds that drive the narrative and environmental immersion in the film. Both score and full soundtrack can provide a lot of power in terms of driving your attention, controlling your arousal, and, filmmakers hope, making you remember the film and the emotions you felt while watching it. But the score and the soundtrack have different ways of accomplishing these goals.

The score of a good film uses music at what it does best—providing an underlying psychological flow to the story without having to give specific information about the action or the environment. One of the most basic techniques is to use a theme song or anthem to provide identification or “branding” of the film or its elements. An anthem uses properly composed music to ensure strong emotional association with the visual and narrative aspects of the film. Think, for example, of the low
drums and forbidding horns of the “Imperial Death March,” which accompanied Darth Vader wherever he walked. When you heard that music, you knew there was trouble.

Music is one of the strongest associational tools available that doesn’t use a bunch of levers, lights, and electrical shock equipment. What is often the first thing you remember about a show you saw when you were young, even if you can’t quite put your finger on the name? It’s usually the theme song. A successful musical theme grabs your attention and locks you into the whole experience of the film or TV show in a short time, usually under seven notes to make best use of your limited short term memory span. For example, if you were born after 1955 and before 1975 and I even mention the 1960s television series
Batman
, I bet the back of your brain starts singing “na-na na-na na-na na-na na-na na-na na-na na-na na-na Batman!” in the appropriate key. And unless you are a serious classical music listener or musician, when you hear five slow notes accompanied by images of apes worshipping a strange black obelisk, you think
2001: A Space Odyssey
rather than Strauss’s
Thus Spake Zarathustra
. Even more minimally, think of the two alternating notes that made up the basic theme to the classic spaghetti Western
The Good, the Bad, and the Ugly.
The brilliance of Ennio Morricone’s score for this film was that not only do you think of that film when you hear this three-second clip, but by using different instruments (a flute, an ocarina, and human voices) consistently before showing different characters, you form an audio and emotional association with the characters even within the context of the story just based on the timbre, the fine structure of the tones, separate from the actual pitch. Perhaps the ultimate in the use of anthemic music was John Williams’s score
for the
Star Wars
series. He used no fewer than eight separate anthems just in the initial film (
Episode IV—A New Hope
) and twenty-six separate ones across all six films.

In all of these, the basic strength of the theme or anthem is based on the limited number of sounds, the relative simplicity of the interval arrangement, the consistency of presentation, and the proper use of repetition (e.g., as an acoustic bracket at the beginning and end of
2001
; when the appropriate characters appeared on camera in
The Good, the Bad, and the Ugly
and
Star Wars
; and every week, same bat time, same bat channel, for Nelson Riddle’s
Batman
theme).

Another effective and well-known anthem in a movie was used in 1975 in the film
Jaws
. I’m betting that as soon as you saw the title, you started hearing that bass ostinato, a low-pitched heartbeat-like sound that grew in loudness as it repeated; deep down in your brain stem, you
knew
something was going to happen that would not involve butterflies and unicorns. The
Jaws
theme, written by John Williams, is one of the great iconic film score themes, and it’s played on, of all things, a tuba. I get a little cognitive dissonance when I think of that; how could a tuba be threatening unless it was thrown at you? I had a friend who played the tuba professionally, and while she was very good, watching her turn purple as she blew into the mouthpiece while wrapped up in eighteen feet of brass and valves was about as threatening as someone having an asthma attack in a radiator (she eventually switched to electric guitar and rocked out). But the tuba has several things going for it as an emotional driver, at least when it’s not relegated to the oompah section of a marching band. The tuba is a very low-pitched instrument, with the bottom end of some models going down into the infrasonic
region.
³⁸
Perceptually, lower-pitched means larger. This is based on very basic biomechanical and evolutionary principles that work in almost all vertebrates. A larger animal will have a larger vocal apparatus and larger lungs to make louder and lower-pitched sounds. In the world of females’ mate selection (like the bullfrogs we spoke of earlier), a larger mate likely will have healthier genes to pass on, or better survival skills at getting resources if it is a social animal.
³⁹
But outside the context of mate selection, an animal whose vocalizations are louder and deeper in pitch than others’ is something that’s bigger than you and is not worried about trying to conceal its presence. In fact, one theory of why large animals roar is that the sound deliberately induces a startle in their prey, to get them to move and hence start the chase, instead of the larger animal having to sit and wait them out.

So using a tuba for the opening theme of
Jaws
was a perfect example of how sound should be used in film, drawing on basic psychophysical principles as well as higher-order associations. The basic structure is a slow heartbeat-like pattern, speeding up until it matches a racing heart. Being exposed to an auditory signal that resembles important biological patterns can induce what is called “auditory facilitation.” An example is an experiment I ran a few years back in a small gallery-like space, in which I played pink noise pulses (with long ramp-on and ramp-off times to make them sound like breathing) very quietly through hidden speakers and monitored the breathing rates of everyone who entered
the room.
⁴⁰
Consistently 80 percent of the people exposed to this simple environmental change matched their breathing rates with the sounds they heard. It follows that if you play a respiratory or heartbeat-type pattern, first slowly and then faster and faster, listeners’ breathing or heart rate too will start racing. So thirty seconds of slowly speeding-up tuba was a perfect way to get an audience to the edge of their seats, wondering what was really going on, when all they were looking at was a woman floating in the water. In short, a good anthem is one that obeys the psychophysical rules for auditory association and learning.

But unless you have a very rigid schedule and spend a lot of time composing specific playlists, real life rarely has a musical soundtrack. What you have out in the world is an environment where the sound provides a background to your daily life, giving you reverberant spatial cues about the size of the space, event-based sounds such as rising winds to tell you that the weather is about to change, and traffic sounds to give you warning about areas to avoid in order to make it to your next destination. It rarely includes someone popping up from behind a hill to play a sad trombone sound when you drop your ice cream cone. While hundreds of millions of dollars are spent on trying to make films more immersive, using everything from surround sound and bass cannons to 3-D visual technology, seeing a movie or watching a TV program is still a limited sensory experience.
⁴¹
The director is trying to create an entire world using
only two of your five senses. There are numerous tricks that can be used, such as distorting the motion at the edges of the screen, which causes your peripheral vision to trick your vestibular system into thinking that you’re inside a swooping fighter jet. But you still can’t smell the dust of a desert, taste the martini the lead character is sipping, or touch the fur of the dog on the screen. Until we build holodecks, a film is limited to your tele-sensory systems, and so the use of well-composed, artfully designed, and skillfully mixed background sound is critical in enabling the suspension of disbelief by enhancing viewers’ spatial and emotional responses.

The heart of this is the use of sound effects. Most people who think about sound effects think about the work of Jack Foley, who in 1939 developed mechanical and analog sound effects to be synchronized with on-screen action. But the first major definitive work on sound effects was printed several years before Foley came on the scene, in the BBC’s
Radio Times
yearbook for 1931. This article explained how to properly use sound effects in radio dramas, and despite being written eighty years ago, it defined the different categories of sound effects in a remarkably scientific manner, spanning everything from realistic event-related sounds to emotionally evocative ones. The parameters were widely adopted in various forms in the United Kingdom and United States and are still used to this day.

But these guidelines didn’t emerge solely from the creative minds behind the radio shows. There was a major wave of scientific discovery in psychoacoustics in the period immediately before the widespread adoption of radio theater, the most commercially important form of media available in homes before the advent of television in the 1950s. The 1920 and 1930s
were a period of major upheaval in the science and engineering of sound, providing not only the infrastructure that allowed the relaying of radio into private homes but also enough of an understanding of the psychophysical basis of sound perception to let people imagine entire story-based environments even through a small, frequency-limited mono speaker. During this period Georg von Békésy published major works on the propagation of sound in space and how sound gets distorted in both the environment and the ear. Harvey Fletcher, the father of stereo recording, carried out many of his basic experiments in speech intelligibility, examining the perceptual basis of loudness and noise rejection and leading to the theory of critical bands. Three Bell Laboratory scientists—Vern Knudsen, Floyd Watson, and Wallace Waterfall—gathered forty other physicists and psychoacousticians and formed the Acoustical Society of America. The 1920s through the 1940s were the heyday of understanding how we perceive the acoustic world, and its most obvious contributions were arguably not the wealth of scientific papers but rather the ability to actually use sound to create worlds through popular media.

There are some moments in film that have stuck with me—something grabbed my attention, made me feel a certain way, or in some cases made me jump out of my seat. I usually find after a second viewing that it’s because something very cool or odd happened with the sound.
⁴²
A properly designed soundtrack,
combining the film score, dialogue, and sound effects, has to weave together the psychophysics of spatial perception; ensure proper temporal alignment of visual and auditory events to get multisensory convergence; provide dialogue to propel the narrative along; and use music and evocative sounds to move the audience’s emotions. The cost of failure is an almost immediate psychological glitch in the viewers; for example, the sudden blare of a musical theme grabs your attention and pulls you out of the story, leaving you saying, “What was that?” A friend of mine who does film composition stated it clearly: “The best soundtracks are the ones you don’t even notice are there.” It all depends on what you want your audiences to feel.

An easy approach is to go for an emotionally primitive response in the audience. Think of virtually any of the current crop of action films, ones that seem to spend a good chunk of the film showing explosions, car chases, gunshots, and robots turning into trucks. The soundtracks for these films consist of very loud sudden-onset sounds and rapid-tempo, loud musical scores with a lot of high-powered bass lines and dissonant intervals, with little or no break; in short, the sound is designed to get you aroused and keep you there with trying to split your time between startles, fear, and excitement. At a psychological level, this is the low-hanging fruit of auditory responsiveness. It’s all about sending traffic back and forth between the brain stem and elements of the fear and arousal system, getting your sympathetic nervous system geared up for fight, flight, or sex (the last of these being the polite term for the third “f” of sympathetic activation). But the problem is that the only way to extend this response over the two hours of a film is to continue making the soundtrack louder and more dissonant, leaving you
at the end of the movie all wired up, but missing quite a few hair cells.
⁴³