Emotion Recognition in Ads: What AI Can and Can't Measure
AI-based emotion recognition for ad testing has come a long way, but it has real limits. What the technology measures, what it infers, and where it still falls short.
Emotion Recognition in Ads: What AI Can and Can't Measure
AI-based emotion recognition is the technology behind most modern ad testing tools, including Wreltik. It's powerful. It's also frequently misunderstood — both by people who overestimate what it can do and people who dismiss it entirely. The reality is more interesting than either extreme.
What the technology actually measures
Current emotion recognition for video advertising works by analyzing visual, auditory, and structural features of the ad and predicting how a human brain would likely respond. It's not reading minds. It's pattern-matching against known neuroscience data.
The underlying models are typically trained on datasets where real humans watched ads while their brain activity was measured — EEG, fMRI, eye tracking, facial coding. The AI learns which features of the ad (motion, contrast, faces, scene changes, audio characteristics) correlate with which brain responses. Once trained, it can predict those responses for new ads without needing new brain scans.
This is genuinely useful. It means you can get predictions about attention, emotional engagement, and memory encoding in minutes instead of weeks, for dollars instead of thousands.
What it measures well
Attention prediction. Visual salience — what the eye is drawn to — is relatively well-modeled. AI can predict with reasonable accuracy which parts of a frame will capture visual attention and where attention is likely to dip.
Emotional intensity. Not the specific emotion (joy, sadness, anger), but the intensity of emotional response — is something emotionally engaging or not? The difference between a flat emotional response and a strong one is detectable and predictive.
Cognitive load. How hard the brain has to work to process what's on screen. High cognitive load predicts drop-off because the viewer's processing capacity is exceeded. This is measurable from visual complexity, information density, and pacing.
What it measures poorly
Specific emotion categorization. Distinguishing surprise from fear, or amusement from warmth — these fine-grained distinctions are hard even for humans, and AI models are unreliable at them. The tools that claim to detect specific emotions are overclaiming. Intensity of emotion: reliable. Type of emotion: not yet.
Cultural context. An ad that's funny in one culture might be confusing or offensive in another. Emotion recognition models trained on Western datasets don't transfer cleanly to other cultural contexts. The visual features might be the same, but the emotional meaning is different.
Narrative comprehension. Does the viewer understand the story? Do they get the joke? Are they following the argument? AI can't answer these questions yet. It can tell you that a frame is emotionally flat, but not whether the flatness is because the joke didn't land or because the content was designed to be emotionally neutral.
Brand perception. Will viewers associate the emotional response with the brand, or just with the creative? This is the oldest problem in advertising measurement, and AI doesn't solve it. High emotional engagement that isn't attributed to the brand is entertainment, not advertising.
The takeaway
Use emotion recognition for what it's good at: detecting attention patterns, finding emotional dead zones in your creative, comparing the relative emotional impact of different versions. Don't use it for what it's bad at: declaring one ad definitively better than another, predicting exact in-market performance, or replacing human judgment about whether the creative is actually good.
The technology is a second opinion, not a verdict. The best diagnosticians use the test results to inform their judgment, not to replace it.