AI product analyst

MEDvidi is an AI-powered mental healthcare platform setting a new standard for safe, effective, and scalable psychiatric care in the United States. We combine licensed providers with proprietary AI tools to deliver consistent, outcomes-driven treatment for conditions like ADHD, anxiety, depression, and more. MEDvidi's technology automates charting, follow-ups, and treatment planning, freeing providers to focus on patient care while improving efficiency and clinical quality.

• Analyze production behavior of Voice AI Agents and conversational AI workflows.
• Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI-driven conversations.
• Work directly with call transcripts, logs, generated outputs, and operational datasets to uncover patterns and actionable insights.
• Design evaluation frameworks for measuring AI quality, reliability, and business impact.
• Build monitoring systems and dashboards for AI performance visibility.
• Measure the impact of AI systems on operational efficiency, business outcomes, and patient experience.
• Detect quality shifts, anomalies, and emerging failure modes in production environments.
• Partner closely with Product, Engineering, QA, and Operations teams on AI initiatives.
• Support rapid decision-making in fast-changing and ambiguous environments.

• 5+ years of experience in Product Analytics, Data Analytics, Operational Analytics, or analytics roles within complex production systems.
• Strong SQL skills.
• Python for analytical investigations and working with large-scale text datasets.
• Experience working with ML pipelines or AI-powered products
• Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, and AI failure modes.
• Experience building monitoring systems, analytical frameworks, or operational dashboards.
• Strong communication skills and ability to translate analysis into product decisions.
• Ability to work effectively in ambiguous and fast-changing environments.
• Fluent Russian; English level B2 or higher.
• Ability to work CET-overlapping hours.
• Будет плюсом: Experience with conversational AI or AI agents; Experience evaluating LLM outputs; Experience with AI quality metrics and evaluation methodologies; Experience with prompt iteration and AI evaluation frameworks

• High-impact environment: A chance to influence how AI is used in real patient interactions.
• Growth & compensation: Competitive compensation and clear growth opportunities.
• Remote-first: Fully remote, long-term B2B collaboration.
• Health & wellness: Health insurance after the probation period, plus sports & wellness compensation.
• Career development: Career growth, including personalized English lessons via Preply.
• Time off: 19 working days of vacation per year, 4 additional wellness days, and paid sick leave for the first 5 working days.
• Culture: Thoughtful gifts for important life events.