ml engineer for EvalOps platforms

The company is building a new AI evaluation and observability platform, targeting its first paying pilot by month 3 and a commercial beta by month 6.

• Lead architecture and a team of 5–7 engineers;
• Build production AI evaluation, RAG quality, and monitoring systems;
• Own LLM observability, regression testing, and cost/latency tracking;
• Set engineering standards and work with pilot clients.

• 5+ Years of engineering experience;
• 2+ Years leading teams of 3–8 engineers through a full product build cycle;
• Strong production-grade Python backend experience with async, FastAPI, APIs, integrations, PostgreSQL, and Redis;
• Deep understanding of LLM evaluation, including LLM-as-judge, calibration, golden datasets, regression testing, and hallucination detection;
• Production RAG experience with ingestion, chunking, embeddings, retrieval, re-ranking, and vector databases;
• Experience with AI agents and agentic workflows, including ReAct, Plan-and-Execute, and supervisor/sub-agent patterns;
• LLM observability and distributed tracing experience;
• Ability to make and justify architecture and build-vs-integrate decisions;
• Strong written technical English;
• Comfortable working in an ambiguous, early-stage product environment;
• Nice to have: OpenAI, Anthropic Claude and open-weight model experience, MLflow or other experiment/dataset versioning tools, EU AI Act / NIST AI RMF knowledge, GDPR / PII handling in AI pipelines, AI red teaming, jailbreak and prompt-injection testing.

• Remote work is available from Da Nang or Hanoi, Vietnam.