In her TED talk last year, Affectiva’s co-founder Rana el Kaliouby predicted: “Five years down the line, I think all our machines are going to have an emotion chip, and we won’t remember what it was like when we couldn’t just frown at our device and our device would say, ‘Hmm, you didn’t like that, did you?’” Kaliouby is the Egyptian-born co-founder and chief strategy and science officer of Affectiva, the MIT Media Labs spin-off which partners closely with Millward Brown and in which Kantar made a US$4.5 million strategic investment in 2011.
Affectiva’s technology – called Affdex – enables computers and other devices to measure emotions by analysing facial expressions; it originally grew out of a research project to help people with autism, and is now the world’s market leading product at gauging consumer emotions in advertising tests.
In the 1970s, psychologist Paul Ekman developed the Facial Automated Coding System (FACS) – an ‘atlas’ of emotion which classifies more than 10,000 facial expressions.
Kaliouby’s breakthrough was to develop a program that automates the hitherto cumbersome coding process, making it practically useable at scale.
Affectiva has since added a layer of analytics and insight onto that process, and has incorporated this into a range of solutions. With the help of a simple webcam, your device – computer, tablet, mobile – can rapidly analyse your facial expression moment by moment to map your emotional reactions on a time line.
Foremost among the applications is Affdex’s integration into Millward Brown’s LINK creative development tool used for ad testing. Graham Page, head of Millward Brown’s global research solutions, says: “It allows us to see viewers’ unvarnished emotional reactions to brand videos, and so allows our clients to see if their campaigns are really connecting with the target audience.”
Millward Brown has researched more than 15,000 ads using the approach so far, says Page. “It enables us to understand how the structure of a video is working, and so give much richer guidance for campaign optimisation. Affectiva’s technology leads the emotion-sensing industry Affectiva, and the wealth of emotional data we now have is a fantastic source for future learning about marketing effectiveness.”
What differentiates Affectiva from competitors, says CEO Nick Langeveld, “is the scientific rigour and analysis that we apply. Others have a general ability to offer a face tracker, and to map facial features into emotional states. But there’s much more to it than that in terms of being as accurate as possible about what those expressions truly mean. That’s where the vast amounts of data and science leadership we have sets us apart.”
Gabi Zijderveld, the company’s VP of marketing and product strategy, says that advertisers use Affectiva’s emotional analytics for three main reasons: “To help optimise the content, to better accentuate the brand engagement they are looking for, and to determine where to allocate the media spend.”
An example of content optimisation was a commercial for Unilever’s Clear shampoo in Indonesia, in which a motorcyclist takes off his helmet to reveal dandruff. A week later after using the shampoo his dandruff is gone. “When the client looked at our data, they realised that viewers were confused, so by tracking the moment by moment analytics they were able to re-architect the story flow to make sure there was no longer confusion,” she explains.
In another campaign for Kellogg’s Crunchy Nut cereal, the client was able to identify which of three executions performed best on repeat viewings, and allocated the budget accordingly.
Television content, including audience testing of pilot shows, is another media-related area where Affdex is being used.
“A couple of examples in the B2B space where it is being used are in online recruitment and legal depositions,” says Langeveld. “Automotive companies are also using the technology to build their next generation automotive experience, using our technology to help understand the driver’s face.”
One example is in sensing when a driver is feeling drowsy and taking corrective action; another is that if a car is stuck in traffic and the passengers are getting frustrated, an alternative route may be found. “The infotainment system in a car can become more like a concierge service, offering suggestions without being asked,” says Zijderveld. Gaming is seen as one of the most exciting opportunities of all. “What if the game could adapt and respond to the emotions of the player?” asks Zijderveld. “For example, it could create a difficulty level based on how you are responding, or change the narrative. Emotional measurement can help the game publisher to know when a player is about to drop out, or when is the ideal moment to offer them in-game purchases.”
There’s virtually no limit to where this technology can be used and the race is on to bring new applications to life that will fundamentally change our relationship with technology. As Kaliouby put it in her TED talk: “By humanizing technology, we have this golden opportunity to re-imagine how we connect with machines, and therefore, how we, as human beings, connect with one another.”
Editor’s Notes
* This article was first published in The WIRE Q2, 2016, issue No. 59;
* To contact the author, or to know more information about ad testing and brand consultancy, please contact us;