machine learning engineer for audio model evaluation

Cantina Labs is a social AI company developing advanced real-time models focused on expression, personality, and realism. Its flagship Cantina platform helps people tell stories, connect, and create through social AI.

• Design model evaluation pipelines for models in development and production;
• Design user studies for subjective model evaluations;
• Convert requirements into measurable metrics;
• Design and develop automated evaluation dashboards to assess model performance and compare results;
• Train new models to capture new and different evaluation metrics;
• Communicate with the model team to help design better models based on evaluation results;
• Communicate with the data team to help determine the data needed to improve model performance;
• Communicate with the product manager to ensure product requirements are measured correctly;
• Help grow the evaluation team as its founding member;
• Lead the evaluation team in the future.

• Strong experience and intuition in designing metrics that capture model performance;
• Strong experience designing user studies on Mechanical Turk or similar platforms;
• Strong experience with model training and fine-tuning for model evaluation;
• Strong statistical knowledge and experience statistically comparing evaluation results and making decisions;
• Very strong engineering and programming skills;
• Experience training ASR and TTS models;
• Experience working in ML teams on large-scale machine learning problems, including models over 3B parameters and more than 1M hours of data.

• Annual base salary range: $200,000-$220,000 (€170,000-€190,000);
• Competitive salary and generous company equity;
• Medical, dental, and vision insurance with 99.99% of premiums covered by Cantina;
• 42 Days of paid time off, including 15 PTO days, 10 sick days, 15 company holidays, and 2 floating holidays;
• Generous parental leave and fertility support;
• 401(K) retirement savings plan;
• Lifestyle spending account of $500/month;
• Complimentary lunch and snacks for in-office employees;
• One Medical membership.