Moodboard / Alamy

A face doesn't speak for itself

Law enforcement strategies rely on the idea that facial expressions reveal emotional states. What if that's not true?

April 13, 2014 7:00AM ET

It is common sense — espoused by “Sesame Street” and psychology textbooks alike — that humans have distinct emotions, each with characteristic expressions. When you’re angry, you furrow your brow and yell. When you’re sad, you frown and cry. Charles Darwin hypothesized that human emotions have evolved just as physical features have, and the psychologist Paul Ekman, known for his work on microexpressions, has traveled the world showing that people everywhere recognize the same facial movements as expressing the same emotions — anger, disgust, fear, happiness, sadness and surprise.

A few psychologists, including Lisa Barrett of Northeastern University, are upending this view. In two new papers, she and her collaborators test the notion of universal emotional expression among a group of people living in a remote region of Namibia. Their results, showing big cultural differences in emotional recognition, have strong implications for therapy, the law, business and even national security.

Foreign faces

In one set of studies, published in Psychological Science, a team led by Maria Gendron, a member of Barrett’s lab, had participants listen to vocalizations for nine emotions (for example, laughter for amusement and a scream for fear) and think of a single word to describe each feeling. The researchers tested two groups of participants: visitors to the Boston Museum of Science and members of the Himba ethnic group in a mountainous part of Namibia. The Americans guessed all nine emotions better than chance. But the Himba succeeded only with amusement. Their accuracy fell below 5 percent for seven of the nine emotions (anger, disgust, relief, sadness, sensory pleasure, surprise and triumph). Even something as simple as screaming “Woohoo!” in a moment of exulting appears to be culturally specific.

In a study published in the journal Emotion, again led by Gendron, American and Himba participants interpreted facial expressions. Participants received 36 photographs of African-Americans making expressions that were meant to depict five emotions (sadness, anger, disgust, fear, happiness) and neutrality. Participants were asked to sort them into as many piles as they liked, as long as every face in each pile depicted the same emotion; participants then labeled the piles however they wished.

Americans sorted their piles into smiling (happy), scowling (angry), wide-eyed (fearful) and neutral faces, with the pouting (sad) and nose-wrinkled (disgusted) faces mixed together. Himba participants made separate piles of happy and fearful faces, but formed groups for the rest that were incoherent, according to American categories. And they tended to label the piles using descriptions of the facial actions rather than mental states. “We demonstrated that facial expressions are not universally recognized in discrete emotional terms,” the researchers write.

Context is key

Serena Williams at the 2008 U.S. Open, after beating her sister Venus.
Matthew Stockman / Getty Images

So how do we understand people’s facial expressions in the real world? By looking at context and constructing an interpretation. “We’re not recognizing emotion in another person. We are perceiving it,” Barrett told me. In one paper she includes a close-up photograph of Serena Williams with an agonized look on her face. You’d think she just stubbed her toe. Zoom out and you see she’s pumping her fist. Suddenly you are perceiving something very different: triumph.

Barrett says her team’s results are evidence for her conceptual-act theory of emotion, in which emotions are not natural categories with distinct signatures in the brain and behavior but rather “constructed events that arise in the moment from a set of more basic ingredients.” Components such as arousal and a positive or negative mood are situationally and linguistically filtered to form an experience of a particular emotion, and such interpretations are malleable. One recent study found that subjects could improve their karaoke performances by reappraising their anxiety as excitement; they simply said to themselves, “I am excited.” Controversially, Barrett believes the emotional palette itself (not just facial and vocal expressions) can differ among people and cultures. Furthermore, that the Himba used labels related to actions rather than mental states to describe their piles of photos suggests that they have not just different emotional categories but a different idea of what emotion is.

Maybe that flicker of the eyes is completely unrelated to the malignant emotion you’ve been primed to spot.

Psychologists have argued that emotions evolved to serve many functions, one of which is to signal our beliefs or intentions to others. Anger, for instance, can act as a threat. Barrett’s work doesn’t upend this evolutionary view, but it shows that there’s some flexibility in how we learn to express our internal states.

Why the grimace?

It’s important to face up to the fact that not everyone expresses and recognizes emotion in the same way. “The applications are endless,” Barrett says. “Anytime you’re interacting with someone, you’re trying to infer their intentions.” Picture yourself on a first date and the person across from you wrinkles her brow. Without knowing her well, you might think she didn’t like your joke, when she meant to convey thoughtfulness (or was noticing that the wine is corked). Playing poker, you think you know an opponent’s hand based on his pursed lips, whereas his mother could have told you otherwise.

Therapists, too, need to understand what’s going through their clients’ heads to interpret their emotional expressions correctly (just as patients need to know that about their therapists). They may also need practice associating a particular person’s expressions and vocalizations with particular mind-sets.

Business partners from different cultures could construe expressions differently as well. Lifted eyebrows might mean one thing in the U.S. and another in Japan or Russia. “The world is only flat,” Barrett says, “if you can read the emotions and intentions of people from other cultures. And in order to do that, you have to know something about the language and customs and culture of those other people.”

The consequences for national security might be most telling. Law enforcement agencies have been trained in techniques to help detect deception, anxiety and criminal intent. But they may be focusing too much on facial or body language and not enough on the scene around the suspect (let alone his individual background). Maybe that flicker of the eyes is completely unrelated to the malignant emotion you’ve been primed to spot.

In fact, in November the U.S. Government Accountability Office released a report that stated that the Transportation Security Administration’s program for screening passenger behavior produces results that are “the same as or slightly better than chance.” According to the review, the TSA has wasted about a billion dollars of taxpayer money. The problem is that it based its program too heavily on Ekman’s research. Recall the difficulty in reading Williams’ emotions from just her face. If you see an airline passenger grimace, is he anxious about his planned attack? Did he just stub his toe? Has he forgotten his toothbrush? Is his underwear bomb starting to chafe? Is he recalling a lost poker game? You can’t know.

“A face doesn’t speak for itself,” Barrett says.

Matthew Hutson is a science writer and the author of “The 7 Laws of Magical Thinking,” about the psychology of superstition and religion.

The views expressed in this article are the author's own and do not necessarily reflect Al Jazeera America's editorial policy.

Related News

Law & Justice, TSA

Find Al Jazeera America on your TV

Get email updates from Al Jazeera America

Sign up for our weekly newsletter

Get email updates from Al Jazeera America

Sign up for our weekly newsletter