Member of Technical Staff — Model Optimization and Inference (New Grad)
Position Summary & Responsibilities
About Nuance Labs
Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person.
We're a research company, with PhDs from MIT, UW, Oxford, CMU, and Johns Hopkins, and industry experience from Apple, Meta, Amazon AGI, and more. Backed by Accel, Lightspeed, South Park Commons, and NVIDIA, we combine frontier research with ruthless engineering needed for consumer-grade, real-time systems. The team is small, the work is real, and the problems are unsolved.
How Nuance Differentiates
Most conversational AI avatars today are hacks — a face slapped on a speech-to-speech pipeline, stuck in the uncanny valley: emotionless, mechanical, one-turn-at-a-time. Current systems take 2–5 seconds to respond; natural conversation requires sub-500ms. That's a 10x improvement, and it demands rethinking the entire stack.
That rethinking starts with full-duplex: an AI that listens and speaks simultaneously, perceives emotion in real time, and responds with a face that actually reflects it. It's an extremely hard problem, and we're developing foundation models designed for it from the ground up.
About the Role
We can train a great model. The next problem is making it fast enough to actually use in a real-time conversation — and that gap is enormous. A model that responds in 3 seconds is a demo. A model that responds in under 500ms is a product.
We’re looking for someone who’s excited about taking trained models and squeezing every last millisecond out of them. You understand — or want to deeply understand — the full stack from model weights to serving infrastructure: quantization, KV cache optimization, kernel-level acceleration, batching strategies. You’ve worked with vLLM, SGLang, or similar frameworks (through coursework, research, internships, or open-source) and have opinions about where they fall short.
This posting is aimed at early-career engineers finishing or recently finished with a BS, MS, or PhD. We don’t require a PhD — we care about systems intuition, engineering chops, and the appetite to go deep.
Our stack is more complex than a standard LLM deployment: we’re serving a full-duplex multimodal system that must satisfy strict real-time latency constraints. There’s a lot of unsolved optimization work here, and we want someone wh
Key Role Competencies & Evaluation Highlights
Evaluators scrutinize rapid product iteration, advanced AI/ML fluency, and the ability to build intuitive, user-centric conversational intelligence interfaces.
Apply for Member of Technical Staff — Model Optimization and Inference (New Grad)
Setup your contact terms & referral note to get recommended directly to the hiring lead at Nuancelabs.