First impressions—the face bias

Lynne Malcolm: When you first get a glimpse of someone's face, how quickly do you make your mind up about them, about what sort of a person they really are? You may be surprised.

Here's singer songwriter Nick Cave and his first impressions of his wife Susie Bick. It's a vignette from the poignant doco about him, One More Time With Feeling.

Nick Cave: The first time I saw Susie was at the Victoria and Albert Museum in London. When she came walking in, all the things I'd obsessed over for all the years, pictures of movie stars, Jenny Agutter in the billabong, Anita Ekberg in the fountain, Miss World competitions, Marilyn Munro and Jennifer Jones and Bo Derek and Angie Dickinson as Police Woman…Jackie O in mourning, Tinkerbell trapped in the drawer, all the continuing, never-ending drip-feed of erotic data came together at the moment, in one great big crash-bang. And I was lost to her. And that was that.

Lynne Malcolm: Hi, it's All in the Mind on RN, I'm Lynne Malcolm. Today you'll hear about the irresistibility of first impressions.

Alexander Todorov is a professor of psychology at Princeton University. He's one of the world's leading researchers on how we're influenced by each other's faces.

Alexander Todorov: One of the very first surprising findings while we were conducting research on first impression is how little information we need to make up our minds. In the very first study that we did we would present faces for 100 milliseconds, 500 milliseconds to a full second, and ask people to make various judgements. I actually thought that to make complex judgements like trustworthiness or aggressiveness, in contrast to attractiveness, you would probably need a long exposure time to faces, perhaps a second. And it turns out that the judgements were fairly indistinguishable from each other. And the main effect of this additional exposure was to increase our confidence, the confidence in judgements. And there have been a dozen of subsequent studies replicating this effect, and now we know that essentially you don't need more than a 200 millisecond exposure to a facial image. So literally these are single-glance impressions.

Lynne Malcolm: Think of this; if you only get less than one second—one second—to make an impression on someone, how accurate can that be? Isn't it superficial? And what are the repercussions?

In his recent book, Face Value: The Irresistible Influence of First Impressions, Alexander Todorov begins with this story about how faces might even influence the outcome of elections.

Alexander Todorov: Every now and then whenever the important elections come, somebody rediscovers the research that has been done in my lab years ago, and then I get lots of interviews, with the most obvious question; well, who's going to win the elections? And in this particular case I had this extremely polite crew from a Korean TV station but they kept pushing in my face very large colour portraits of Korean politicians who were running I believe for president, and they were asking me to say, well, who is the person who has the highest chance of winning? And I always politely decline to make these kinds of guesses.

Lynne Malcolm: Faces are probably the most important social stimuli. In fact in the first year of life, babies look at the faces of caregivers more than any other object. But the idea that you can read someone's personality from the shape of their face seems to go a bit far, don't you think?

However, this idea was the basis for the science of physiognomy, which dates back to Aristotle's time. It was made popular in the late 18th and 19th centuries with the publication of beautifully illustrated books showing facial features which were said to align with certain character traits.

Is it really possible that personality can be accurately perceived from faces?

Alexander Todorov: In fact this was the premise of physiognomy and in some strange way I think that there is a revival of physiognomist ideas under different guises. There has been a number of papers claiming that people can guess from facial appearance all kinds of things, like political orientation, sexual orientation. There have been computer science studies claiming that presumably you can guess one's criminality from their face. And I think this is why the history of physiognomy is very important because the same kind of blunders that have been committed 150 years ago are happening today too.

Lynne Malcolm: So are there particular times or phases of history where we are more likely to jump to those conclusions, those first impressions?

Alexander Todorov: Well, I think yes, I'm mainly speculating because it's hard to come up with hard evidence but I really think that physiognomy emerged at the time when you have modern societies, which is only in the last 10,000, 15,000 years of human history where you suddenly have to live with lots of strangers. Jared Diamond had this great line that this was the first time in the history of humanity where we had to encounter strangers without trying to kill them.

And it's not surprising to me that physiognomy become so popular in the 19th century because this it's the time of big industrial migration. The population of major cities like London and Paris quadruples, and suddenly you have lots of foreign human beings, strangers that often you don't even share the same language, and it is a bit scary. So physiognomy almost gives you the illusion that you can size up people, you can classify them in human types. And in some ways you can based on appearance, what kind of clothing they are wearing, and all kinds of other social symbols that indicate social status, social class and so on.

Lynne Malcolm: But there is no scientific basis for being able to tell a personality simply by their face?

Alexander Todorov: There is very little. So let me qualify this. We can tell a lot of things about the person in the moment of here and now. We can tell their mental states, we can tell their emotional states, we can often tell whether the person is sick, we can tell whether the person is sleep deprived. So there are a lot of things that we can tell that reflect momentary states.

It is also the case that over time, life leaves traces on our faces. For example, it turns out that if you look at the elderly above 70, the biggest predictor obviously of survival or mortality is their biological age, but the next is a big one, which is bigger than anything else that physicians have found, is how old you look relative to your biological age. So people who look younger, they tend to live longer. At first sight that seems very surprising, until you look at, well, what are the determinants of looking younger?

Well, number one is genetic luck. But then the other things are having a better life, coming from a higher socio-economic status with more wealth and better access to healthcare, not having a chronic condition, not smoking, not being chronically exposed to sun. So all of these things that contribute to your health, they contribute also to how you look. The question is, is this something about the character of the person or their life circumstances?

And there are other things you can tell from the face. For example, larger bodies come with larger faces, so people can guess physical strength, they can guess body size. But is that about character? The judgements might be accurate here and now but they are a very, very lousy guide of what the person is across time and situations.

Lynne Malcolm: As you heard, Alexander Todorov and his colleagues have shown that you don't need to be exposed to an image of a face for much more than 200 milliseconds to make a judgement about the person. Here's how they study this experimentally.

Alexander Todorov: When we're interested to what extent how little information people need to form a judgement, we can flash different faces, this could be natural faces or they could be synthetic faces generated by a mathematical model, and we can present the face, let's say, for 50 milliseconds, and then we perceptually mask it. That means that it has to be replaced with a different image, so it's effectively presented for 50 milliseconds and we can present it either to the same participants or different participants for a longer exposure, and essentially we are looking at how much these judgements, after a limited time, agree with judgements which we have unlimited time to look at the faces. That would be one way.

Another way is we can actually build mathematical models that visualise different impressions. That is, they visualise our face stereotypes, our stereotypes for trustworthiness, dominance, competence, extraversion and so on. And then we can generate faces from this model and again present it very briefly to participants, ask them to make judgements, let's say, of trustworthiness, and to the extent that they evaluate faces that were generated from the model to be untrustworthy as untrustworthy, this tells us that they are picking up on this information.

Lynne Malcolm: So let's just take one characteristic; trustworthiness. What does a face look like if it's trustworthy?

Alexander Todorov: So face perception is fairly complex, and it's kind of hard to describe in verbal terms exactly all of the changes that go into an impression of trustworthiness, and that's why it helps when you can visualise what goes into the impressions. But we can describe at least the most global attributes, characteristics that are immediately salient when you build the model of impressions of trustworthiness. And one of the major inputs is emotional expressions. So faces that kind of look happy are perceived as more trustworthy. They don't need to be grinning, it could be very subtle expressions, but they are perceived as more trustworthy. Faces that kind of look disgruntled and somewhat angry are perceived as untrustworthy. In many ways you can think of this as a sort of a meaningful input to the impression of what we are feeling right now.

The second input is really femininity and masculinity. So this goes along with a lot of gender stereotypes, and we tend to trust and like more feminine faces than more masculine faces. So these are two of the biggest inputs to impressions that we can capture with our mathematical models.

Lynne Malcolm: You mentioned faces appearing to be more feminine are considered more trustworthy. What makes a face look feminine?

Alexander Todorov: So, one, it's obviously the shape of the face, for example the chin, the eyebrows change, and they are different for men and women. But the other thing is the darkness of the skin. Within every culture that have been studied so far, if you compare men and women, generally the male skin tends to be darker than female skin, so that's a good signal for gender.

And then with very few slight alterations you can make the face look either like a male or a female. And in fact all of the alterations have to do with increasing the contrast between the eyebrows, between the eyes and the mouth and the rest of the face. And this explains why makeup is so popular, because makeup increases this contrast and makes faces look more feminine.

Lynne Malcolm: And how does our first impressions of feminine characteristics tie in to gender stereotypes? What impact does it have on people who may have a more feminine looking face?

Alexander Todorov: They go quite along the gender stereotypes. So, for example, although we perceive feminine faces as more trustworthy, we perceive them as less dominant. And if you think that the dominance is an important characteristic of a leader, then women would be less likely or people with feminine faces would be less likely perceived as strong leaders.

Another interesting thing is if you look at competence, which is a very, very important characteristic for all kinds of social demands, for job success, if you look at specifically impressions of competence, the major components are attractiveness. So competent faces are much more attractive looking than incompetent faces. But if you can actually remove attractiveness, which we can do because we have mathematical models of these impressions, what is left, it's a masculine, super-masculine face, a male face with a confident look. So you can say, well, even this impression has aged hidden gender bias because you don't see it immediately that it's related to femininity and masculinity, but you can see it after you remove attractiveness. So in a sense, attractiveness of women will help in many of these situations, but at the same time, the other part of the stereotype for competence is really masculinity and expression of confidence.

Richard Nixon [archival]: I made my mistakes. But in all of my years of public life, I have never profited from public service, I have earned every cent. And in all of my years of public life, I have never obstructed justice. I welcome this kind of examination because people have got to know whether or not their President is a crook. Well, I'm not a crook. I've earned everything I've got.

Lynne Malcolm: Is there any particular part of the face, like the eyes or the mouth, that we look to most often?

Alexander Todorov: This is actually very interesting. Often our intuitions are wrong when it comes to face perception, and an example is with the eyes because if you ask people what is the most important feature of the face, and most people will say, well, of course the eyes, but it turns out that for recognising emotional expressions, the eyebrows are much more important than the eyes, simply because they move to a much larger extent than the eyes. And in fact even for recognition of individuals, familiar individuals, the eyebrows are more important than the eyes. Imagine Nixon without his eyebrows, he becomes virtually unrecognisable, and if you just remove the eyes but you leave the eyebrows, he is perfectly recognisable. And that doesn't work only for people like Nixon with prominent eyebrows, it works with lots of familiar faces.

Lynne Malcolm: Alexander Todorov, leading researcher in the science of first impressions, from Princeton University.

You're with All in the Mind on RN. I'm Lynne Malcolm.

When it comes to judging people's character by their faces alone, cultural factors play a big role.

Alexander Todorov: For example, cultural typicality is very important. So we tend to trust faces that we perceive as typical or closer to what we think is our typical face, which is obviously a function of our perceptual experience with faces.

Lynne Malcolm: So this would also feed into a racial bias. So we may not trust somebody who doesn't look like a typical face that we see all the time.

Alexander Todorov: Yes, and we've done studies testing this hypothesis. In this particular case we actually focused on two cultures, Israel and Japan, and we created a morphing continuum between what would be a typical Israeli face and a typical Japanese face. And not surprisingly when you ask participants in both countries to evaluate the trustworthiness of these facial morphs, Japanese perceive faces that look more and more Japanese as more trustworthy, and the other way around for Israelis. And that's not the case for attractiveness. Typically faces that are racially mixed are perceived as more attractive.

Lynne Malcolm: So in a lot of ways, inherit in our forming first impressions is a real bias, it's almost like a 'face-ism'.

Alexander Todorov: Yes, very much so.

Lynne Malcolm: So should it be treated like any other bias and should we try and counter it in some way?

Alexander Todorov: I think so. And in fact in some ways I think it's more dangerous than any other biases because, for example, people have been aware and they know that there are lots of biases related to sexual orientation, biases related to race, and once you know and you are aware of this you can design policy that prevents you from committing discrimination based on sexual orientation or based on race. But if you think that first impressions are just fine and accurate, well, we have a problem here because they go sort of unacknowledged, under the table. And in that sense, yes, we should be aware of these kind of biases and try to prevent them from influencing our decisions.

Lynne Malcolm: And have you done some research to look at how we can combat these effects of first impressions?

Alexander Todorov: Generally people will change their minds to the extent that they have good reliable information about the person. The problem is if you have situations where your first impression essentially is the threshold where you decide whether you would give a chance to this person or not. Or if the interaction is pretty ambiguous. To give you an example, with interviews, psychologists have known now for 30, 40 years that unstructured interviews are pretty useless in terms of predicting future performance or professional success, and yet we keep using it. But we put too much faith in these interviews, and they contain much more than just impressions from facial appearance. And in this particular case, if you are aware of the bias in interviews, you should make your decision before the interview based on reliable information, like letters of reference, past performance records in the relevant domain and so on. The interview should be completely irrelevant in that sense.

Lynne Malcolm: So would this be an argument for not including a photograph, say on your CV?

Alexander Todorov: Well, yes and no. It depends on who is the recipient. In fact there was an interesting study by an economist. There is a website in the United States, Prosper, where private citizens apply for loans and other private citizens can give them loans. And this website contains an enormous amount of information about people who are applying for loans, including job history, employment status, credit records, so this is all very good information to decide whether the person will default on the loan or not.

And there were two relevant findings. Number one, not all people post pictures of themselves because you don't have to, but it turns out it's a good idea in this particular case. So people who posted pictures of themselves were more likely to get loans. But the second finding which was much more disturbing is that people who look more trustworthy, they were more likely to get a loan and they were more likely to get a loan with a lower interest rate. So it depends.

You have to be aware that whenever you post an image of yourself, it will be judged. And in that case you should truly think, well, what is the context? Is it a professional website like LinkedIn, or is it something like Facebook for your friends or is it an online dating website. And we know that in fact people may form completely different impressions from different images of the same individual. And this is important to know and it's another strong reason not to trust first impressions because a different image of yourself can generate completely different impressions. And it's not very hard, you can just take a second to reflect on the many pictures of yourself…now in our digital culture you have thousands and thousands of images, and we don't look equally well in all of them.

Lynne Malcolm: And I guess possibly a more serious context is in the crime and justice field where facial impressions could really determine people's lives?

Alexander Todorov: Yes, I agree, and there have been a couple of real-life studies that suggest that in fact this is the case. There has been many experimental studies, and of course they are not quite from real life, but they are experimentally controlled, and if you show them images of, let's say, a more criminal looking person versus a less criminal looking person, and again, we have models that exist that can manipulate the image of the same person to make it more criminal looking or less criminal looking, people are more likely to judge the more criminal looking person as guilty. And there was a recent study where they looked at all prisoners in Florida who were sentenced either to the death sentence or to life without parole for the same crime. And controlling for age, controlling for race, it turns out that the prisoners, the unlucky ones who were sentenced to death, looked more untrustworthy than the other ones.

Lynne Malcolm: So when you say whether they look more criminal or less criminal, are you referring to their trustworthiness?

Alexander Todorov: In fact it is correlated with two important trait impressions. One is trustworthiness, that is more criminal looking is less trustworthy. It is also correlated with dominance and masculinity, so more criminal looking is more dominant and more masculine. And generally it is really highly correlated with impressions of threat, and threat is a really untrustworthy dominant looking face.

Lynne Malcolm: So how early in our lives do we start forming these first impressions and biases?

Alexander Todorov: Quite early. We've done some studies with colleagues from Harvard University where we had children between three and seven years old and also we had adults, and we showed them some of the faces generated by our model of trustworthiness, and we can simply show a face that is manipulated to look trustworthy and a face that is manipulated to look untrustworthy and you ask simply the kids, well, can you point to the nice person? And just like adults, in about 70% of the cases three-year-olds point to the trustworthy looking face as the nice person. So these judgements appear very, very early on in life.

Lynne Malcolm: This is very unnerving really, isn't it, because we just seem to not have power over the impressions that we form.

Alexander Todorov: In many ways, and again I think what is justifiable for these impressions is that they might be okay in terms of here and now, in this particular case for impressions of trustworthiness, the major input is really emotional expression. So you're better to approach somebody who is signalling that they are happy and hence they are willing to talk to you than somebody who looks angry. But it is disturbing if people think that this is what the person is like in general.

Lynne Malcolm: So how easy is it to manipulate someone's first impressions of you? What can you do to influence someone positively or negatively?

Alexander Todorov: Well, that of course depends on the situation, it depends on the mood of the person, but generally first it will depend whether it's a business interview, for example a job interview, or whether it's a party, you will behave differently, and to the extent that your behaviour conforms to the norms of the implicit rules of the particular situation, that would be good. And generally when we appear to be happy and rested, we will be perceived more positively.

There are interesting studies of taking pictures of people after they have a good night sleep and after they have been sleep deprived. And you can immediately tell the difference. And sleep deprived people, they kind of look somewhat depressed, their skin is paler, so they just look tired. And if you show these two images of the same person to two different groups of people and you ask them to judge the intelligence of the person and their attractiveness, not surprisingly the people who see the sleep deprived image, they think, well, this guy is not very smart, they are also not attractive, they look a little bit depressed. But clearly it's not about the person, it's just about the particular situation in which you find yourself. So generally when you feel good you will look better and people will perceive you better.

There was another interesting study where women were asked to wear either something that they really love or something that they hated. And then while they were wearing these clothes, their pictures were taken, just of the head, head shots, nothing else. And guess what? When they were wearing what they liked, people perceived them as more attractive.

Lynne Malcolm: So, overall, what shift do you think that we should be making to take into account these potentially quite powerful biases that change people's lives?

Alexander Todorov: I think that the first step is to be aware of the potential biases that come from first impressions and to be a little bit less confident in your own impressions, to be a bit more modest, and this is true for all of the judgements and decisions that we make in our life. I often say that getting a degree in psychology is really getting a degree in modesty because you realise that there are so many things that colour our judgements, colour our decisions, and yet we seem to be blind to these viruses. We often think that we are impervious to all kinds of effects and they only apply to other humans but not to us.

So I think the first step is really to be aware that there are many circumstances under which you will form an impression and that might be misleading. So think more carefully, especially if it's consequential, like the hiring of a potential employee, or making a judgement about the person with whom you are going to live for a long time, like if you decide that your neighbour is an untrustworthy looking guy because they just look this way.

Lynne Malcolm: Professor Alexander Todorov from Princeton University and author of Face Value: The Irresistible Influence of First Impressions. Head to the RN home page and the All in the Mind site for more on first impressions, and to see some of the facial images Alexander Todorov was talking about.

Thanks today to producer Diane Dean and sound engineer Mark Don.

I'm Lynne Malcolm, catch you next time. Bye for now.

Copyright © 2017 ABC