SYNC
SYNC is an emotionally-aware 3D avatar AI companion for first-generation college students. Unlike reactive chatbots, SYNC proactively monitors emotional trajectory and intervenes before burnout occurs — making it the first AI companion designed for academic resilience rather than task completion.
The Problem
First-generation college students face 2.3× higher dropout rates than continuing-generation peers, driven largely by undetected emotional disengagement and lack of early intervention. Existing mental health and academic support tools are reactive (crisis hotlines, tutoring) and carry stigma. Students disengage silently before any human support system notices.
The Solution
SYNC implements 10 novel research mechanisms including the Affective Engagement Model (AEM) for emotional state tracking, Temporal Emotional Feedback Fusion (TEFF) for trajectory prediction, and the PantoMatrix audio-driven motion engine for photorealistic avatar expressivity. The system runs fully on-device using Ollama Qwen2.5:7b, ensuring student privacy.
Key Features
Challenges & Learnings
Achieving photorealistic lip-sync and emotion-congruent motion in a browser-based Three.js environment required designing the PantoMatrix engine from scratch — existing open-source solutions (Rhubarb, wav2lip) either required server-side processing or produced robotic motion. Running a 7B LLM entirely on-device while maintaining sub-2s response latency required aggressive context management and streaming token rendering.
Research Contributions
First AI companion architecture explicitly designed for proactive emotional intervention in academic settings. AEM and TEFF together constitute a novel emotional trajectory modeling approach distinct from sentiment analysis. Under IP review.
Business Impact
Targets a 30%+ reduction in first-generation student dropout rates via early emotional intervention. Designed for deployment in university student success centers and community college systems. Privacy-first architecture makes institutional adoption feasible without FERPA concerns.
Future Direction
Longitudinal study with real first-generation students, federated learning for model personalization across students without sharing data, and multi-avatar social simulation for peer interaction practice.
Tech Stack
Quick Links
Want to see more of my work?
Back to Projects