ABOUT THE PROJECT
Lexus AI Emotional Journey explores how video and audio can induce an emotional response in the viewers. Through face tracking technology, the interactive video reacts to the user's facial expression, delivering a unique advert every time.
ABOUT THE ROLE
Working closely with the development team, I developed and defined the whole UX for this experience. Given the nature of face tracking and determining which emotion the user had at each moment, there were many limitations and challenges that were handled by giving the video different sections and letting know the user what was happening behind the scene. The main tasks included defining the whole UX strategy, user flows and rough wireframes.
ROLE
Lead UX Designer
CREDITS
Unit9
The background
Lexus wanted to communicate that their new ES Self-Charging Hybrid model is designed to make you feel your best. So we created the world’s first emotion-driven video experience that responds to how you feel in real-time — inspired by the car that does the same.
The execution
The Lexus ES AI Emotional Journey is an experiential film experience dynamically generated based on your current emotional state. Using AI sentiment analysis, we crafted personalised videos in real-time that mirror the viewer’s feelings, with live data visualisation corresponding to changes. As well as responding to the relevant emotions, the AI also aims to drive the viewer towards feeling more relaxed by the end of the film.
We used MorphCast’s software development kit to read and translate facial expressions. The sophisticated technology detected faces and used a CNN (Deep Convolutional Neural Network) approach to analyse factors like emotion, age, gender, and head pose. Once it determined your current mood — stressed, surprised, relaxed, puzzled, or happy — it adjusted the colours, videos, sounds, and animated graphic elements accordingly to match.
(Credits: Unit9)
