Scientists studying emotional expression now study faces in context so AI designers should too

Sign up for our newsletter, Artificiality, to get our latest AI insights delivered directly to your email inbox.

Last year, new research came out about how the science of emotions isn’t keeping up with the power of the emotional AI technology. Now, Nature magazine has released a handy update on the emerging science of emotions. It includes a challenge to the AI community: “Are AI companies going to continue to use flawed assumptions or are they going do what needs to be done?”

This is important for AI designers to understand. Failing to understand the full complexity, nuance and variability of emotional expression risks both bias and avoidance. People will easily be able to outsmart an AI by faking expressions. Humans do this all the time; in customer service, people smile even though they don’t necessarily feel happy.

The modern approach to emotional analysis from facial expression is to include the highly varied context under which humans express emotions through facial expression. Our facial expressions are far more nuanced and variable than was previously assumed and much of this nuance stems from the context in which we express emotion. There are three main contextual considerations:

  • situation – what’s the background? who is present? what kind of situation?
  • personality traits – what is someone’s internal context?
  • temporal – how does perception of emotion change as someone face moves? what about with acoustic changes in vocalization? how about with changes in body posture?

Scientists studying emotion have shifted their focus to study faces in context, not just faces. This means that AI companies working in emotional analysis should do the same thing.

In practice this means:

  • use moving faces and introduce dynamic analysis not static pictures. Vary the situation.
  • enhance labeling of training video/images to include context.
  • use technology (rather than actors) to discover patterns between facial movements and perception of emotion.

This video from Nature is worth a watch.

Photo by Lidya Nada on Unsplash

At Sonder Scheme, we are on a mission to help humans win in the age of AI. We have created a human-centered AI design system that allows leading companies to create AI in an inclusive way. Companies like The New York Times, Starbucks, R/GA, Galp and the National Headstart Association tap us for engaging presentations at team/executive forums, in-depth workshops or long-term learning journeys. We support them with executive coaching and online access to our design system and up-to-the-minute insights into the frontiers of AI technology and ethics.

Share on email
Share on facebook
Share on linkedin
Share on twitter
Scroll to Top