Embaudy

Initial Brief & Exploration

We were asked to create an interactive design project using creative coding, computer vision, and AI. With such a broad scope, our team:

• Researched installations featuring motion-tracking, emotion-recognition, and immersive public experiences (e.g., marketing campaigns, art performances, and emotional-sensing devices).

• Conducted an affinity mapping exercise to cluster insights, revealing three core themes:

  
  

1. Body & Facial Tracking

2. Interactive Public Installation

3. Emotional/Mental Health

From there, we refined our challenge statement:

“How can we utilize body & facial tracking to create an interactive public installation focused on emotional/mental health among college students?”

Literature Review & Rationale

Data from sources like the National College Health Assessment indicated that 60% of college students met criteria for at least one mental health issue post-pandemic. Anxiety affected 59%, while 48% faced depression (Active Minds Survey). This confirmed a genuine need to address emotional awareness on campus.

User Research

We conducted assumption mapping and a survey with 25 college students, focusing on emotional identification, communication, and existing coping mechanisms. Key findings included:

• Difficulty Expressing Emotions: Students often used vague terms like “good,” “tired,” or “tense.”

• Existing Methods: Many students reflect daily on their feelings using journaling, talking with friends, or creative outlets.

• Desire for Emotional Awareness: Most believed understanding and communicating emotions could positively affect stress management and relationships.

• Nonverbal Stress: Students noticed changes in body language, sleep, and appetite, reinforcing the importance of an interactive, visual approach.

Revised Goals & Constraints

Goals:

1. Help students identify and express their emotions in a supportive, public-facing experience.

2. Encourage open discussions about mental health without positioning Embaudy as a clinical solution.

Constraints:

• Must enable immediate, visually engaging feedback.

• Limited timeframe demanded a focus on feasible tech (TouchDesigner, standard cameras).

• Public deployment required an intuitive, non-invasive interaction model.

Ideation & Prototyping

We used storyboarding and brainstorming (similar to Crazy Eights) to generate interaction concepts:

• Body/Facial Tracking: Real-time color and particle effects responding to gestures and expressions.

• Audio Capture: Visualizing speech volume or sentiment.

• Single-Screen vs. Multi-Screen: Evaluated feasibility; multi-screen proved too complex with limited dev time.

Through quick play-testing with our peers, we discovered:

• Facial Tracking Accuracy: Needed to exaggerate visual feedback for subtle expressions.

• Tech Setup: Kinect Azure was deprecated, so we relied on standard cameras, limiting advanced depth features.

Visual Design & Emotional Mapping

To represent emotional states:

• Color Associations: 40% of surveyed students mentioned colors when describing emotions. We consulted research on color-emotion mapping from sources like the University of Toronto.

• Elemental Motifs: Water, fire, air, and earth informed fluid, organic visuals.

• Particle Systems: Implemented GPU-based particle flows in TouchDesigner to create dynamic, immersive graphics that shift based on user inputs.

Final Prototype

Our initial prototype features a large digital screen in a public area. Users approach, and the camera detects facial gestures and hand movements:

1. Real-Time Visual Change: Colors and particle motions adjust in response to user movement or expression.

2. Emotional Prompt: A subtle legend indicates general emotion categories (e.g., “calm,” “energetic,” “tense”) tied to color shifts.

3. Resource Link: A QR code directs users to mental health resources, encouraging deeper reflection.

Feedback & Iteration

A panel of seven design judges provided critiques:

• Clarify Purpose: Emphasize that this is an art-driven interactive piece, not a clinical diagnostic tool.

• Call to Action: Move the QR code front and center for easier access.

• Make It Fun & Approachable: Lean on playful elements to reduce potential intimidation around mental health topics.

Next Steps

• Usability Testing: Gather on-campus data to see if visuals and interactions effectively spark dialogue.

• Technical & Accessibility Upgrades: Experiment with improved sensors or alternative cameras; consider public installations in varied lighting.

• Collaboration: Partner with industrial designers to refine hardware setup and explore resilience in unsupervised public spaces.

• Refine Business Value: Identify campus partners (e.g., counseling centers, student organizations) who might sponsor or co-promote the installation, ensuring long-term viability.