Designed and prototyped an interactive billboard, leveraging user-centered design methods to enhance mental health awareness on college campuses. Integrated machine learning and computer vision technology to create engaging, innovative user experiences.
Utilize creative coding, computer vision, and artificial intelligence technologies to develop an interactive project.
Something that uses body & facial tracking, that is public and addresses emotional & mental health
We looked 4 different industries: marketing, performance, personal devices, and research.The project brief gave us a lot of room to explore. To begin our process, we conducted individual research to see how creative coding, computer vision and artificial intelligence technologies were being incorporated into interactive experiences.
We combined the three themes we identified to narrow down our focus for further research.
We found that college students would be an ideal user group, since they were a population in need and that population was easily accessible
We found that 60% of college students met the criteria for at least one mental health problem during 2020-2021. Additionally 59% of college students are affected by anxiety, followed by depression at 48%.
We added college students in our challenge statement, so we continue on better understanding our users.
We wanted to better understand college student’s habit and behaviors concerning mental health.
We conducted an assumption mapping exercise to understand what we needed to plan, evaluate, defer, and generate for our project.
We found that college students knew the importance of mental health, but the biggest barrier to accessing help was the stigma surrounding it.
We conducted a survey using Google Forms to better understand college students. We received 25 responses.
We knew enough to follow tutorials to test and prototype interactions and experiences.
In addition to user research, we conducted a technology exploration to better understand the strengths and weaknesses of the currently available technology surrounding real-time interactivity, facial tracking, and body tracking.
We distilled both our user research and secondary research to develop two personas
We collected all the information from our research phase and distilled it into personas. We further narrowed our scope by crafting a problem statement informed by user insights and our personal interests.
We identified that our intervention should happen at the first stage, our aim should be creating a “desire for change” through building awareness.
We wanted to better understand the journey that someone takes in seeking help and or going through a mental health journey.
We knew our medium will be a large digital screen, and we would need to create something interactive.
Our goal was to create a desire of change amongst college students to help start their mental health journey.
Our key pain points were that college students: Lacked a vocabulary to talk about their feelings & Were hindered by social pressure not seek for help
At this stage we wanted to define a clear challenge statement which will be our guide to ideating and developing ideas.
Our winning idea was a fun digital mirror that would show how you felt as you interacted with it your body.
This allowed students to build awareness and express how they feel through their body, removing the need to talk or share about their emotions.
We could measure project success by how many people scan our QR code, this would be an indicator that the experience initiated the first stage of the mental health journey.
With our problem statement in mind, we started our development with an ideation exercise. We storyboarded different interactions we thought would be engaging, similar to a “crazy eights” exercise.
Using our surveys and secondary research we created a way to collorate different emotions with color.
We referred back to our survey and literature review to help us understand how to indicate different emotional states. During our user research, 40% of college students said that colors came to mind when visualizing emotions. We then used a study from the University of Toronto to create a color map for emotions.
Surveys indicated organic like shapes resembled emotions well, so we replicated organic movements and shapes using a particle GPU
Other than color, college students indicated elemental motifs such as water, fire, and air visually resonated with different emotional states. We then created a mood board of different visuals we wanted to create, inspired from the fluid and organic characteristics of the elements. With further testing we were able to get something organic and fluid using a particle gpu.
Our design visualizes emotional states with out the need for talking or verbalizing how we feel. By removing the need for verbal expression, we can connect students with appropriate mental health resources.
We combined interactions to further test in our next phase. We aimed to create an engaging public survey that required users to use their bodies to record input.By using non-traditional interactions, we created a sense of exploration, reflection, emotion, and mental health learning for users. The ambiguous experience encourages deeper engagement.
We had to reduce the whole experience into one screen, which was advantageous since we were able to make the experience more intuitive.
Multilayered screen interactions were difficult since there were no way to save user data from screen to screen. Facial tracking was not accurate and would loose focusWe also tried the Kinect Azure camera for better tracking however it was deprecated and didn’t work with our current devices
Based on user’s facial gestures and body movements, we were able to visualize students emotional state.
We created an interactive billboard that responds to users’ facial gestures and hand movements. When someone approaches, a camera captures their actions and creates a digital copy of their body. A prompt instructs users to move and smile, while a color legend at the top explains how the visuals connect to different emotional states. The more someone moves and smiles, the more the visuals indicate high energy and joy. A QR code at the bottom right provides resources and tracks engagement.
Visually our initial design lacked the fun and inviting look and feel we were aiming for. We redid the interface, and made it more fun and inviting. Referencing trendy aesthetics from headspace and even movies like Inside out, to relate more with our Gen Z audience.
Our project was judged on presentation, process, and execution by a panel of seven design instructors. Their feedback included:
Misinterpretation: The interaction could be misread as an accurate measure of someone’s emotional health.
Ensure the project is not presented as a mental health indicator.
Focus on Fun: Emphasize making this a fun experience for college students.
Clear Call to Action: The QR code’s call to action is not clear and should be more prominent.User Testing: Conduct user testing to further iterate on the design.
Acknowledge and Adapt: We learned the importance of clearly defining the purpose of our project. Although addressing mental health is crucial, feedback indicated our project, Embaudy, was perceived as a mental health solution. We pivoted to emphasize Embaudy as an interactive art/public installation to promote dialogue about emotional well-being. This shift allowed us to reframe the project as a reimagined form of traditional media like flyers and billboards.
Feedback is great! Feedback encouraged us to make the experience more fun. We redesigned the visual components using neumorphism to create a playful digital reflection of college students, drawing inspiration from popular emotional management tools like Headspace and Inside Out. This adjustment made the interaction more engaging and accessible.
Address Technical Challenges: We encountered technical challenges with multilayered screen interactions and facial tracking accuracy. By pivoting to a one-screen layout and exaggerating hand movement data for better visualization feedback, we adapted our approach to overcome these hurdles.
Importance of Usability Testing: Due to time constraints, we couldn’t conduct usability testing. Moving forward, we plan to test the interaction on our college campus, gather data, and iterate based on user feedback. This step is crucial for refining our design and ensuring it meets user needs.
Leveraging Art and Technology: Embaudy demonstrates how art and technology can raise awareness about social issues and explore real-world solutions using the double diamond methodology. This approach has shown us the power of combining creativity and technical skills to address complex problems.
Feedback revealed a significant weakness of our design. During the designing process we can get lost in the technical aspect of the design, but as designers we should always put the user first...If you’re unsure about something your users will tell you
