Data scientist in AI agents

Then Just Eat Takeaway.com might be the place for you. We're a leading global online delivery platform, and our vision is to empower everyday convenience. Whether it's a Friday-night feast, a post-gym poke bowl, or grabbing some groceries, our tech platform connects tens of millions of customers with hundreds of thousands of restaurant, grocery and convenience partners across the globe.

• Build and improve components of our agent systems — prompts and workflows, tool use, and handover to humans. Translate operational requirements into agent behaviour that is reliable and testable.
• Build and maintain evaluation suites for agents: datasets built from real conversations and scenarios, LLM-as-a-judge and deterministic metrics, and end-to-end tests of tool-using agents.
• Run offline experiments to compare models, prompts, and agent designs, and use the results to make clear ship / don't-ship recommendations.
• Help move us towards automated and well-grounded online evaluation.
• Instrument agents with tracing, structured logging, and quality metrics.
• Build and maintain dashboards and alerts covering quality, latency, cost, and failure modes.
• Analyse production traces and historical conversations to find regressions and the next best improvement.
• Ship agent changes safely — well-tested Python, code review, version control. Be mindful of production rules, data privacy and security.
• Use SQL and Python to analyse large volumes of conversation and operational data, define and track quality and business metrics, and quantify the impact of agent changes.
• Translate evaluation and monitoring results into clear recommendations for the team and stakeholders, using data visualisation where it helps.
• Work with Backend Engineers, Operations and Product to expand what our agents can do, and contribute to a growing internal community of people building user-facing agents at JET.
• Spot opportunities to automate evaluation, testing and release workflows, and make contributions to them.

• Hands-on experience building and shipping user-facing AI agents in production — LLM-based systems that real users interact with, where reliability, latency, safety and graceful handover to humans matter.
• Practical experience with agent evaluation: building evaluation datasets, defining metrics, using LLM-as-a-judge, regression testing agents, and running offline and online experiments to compare prompts, models or agent designs.
• Practical experience with agent observability and monitoring: tracing and debugging multi-step agent runs, defining quality/cost/latency signals, and building dashboards and alerts for LLM systems in production (e.g. OpenTelemetry, Datadog, or similar tooling).
• Experience with agent frameworks (e.g. Google ADK, LangGraph, or similar), tool use / MCP, and multi-step or multi-agent designs.
• Strong Python and SQL; good software engineering habits — typing, linting, unit tests, Git, CI/CD, code review.
• A solid data science foundation: statistics, experiment design, and the ability to reason about whether an observed change is a real improvement.
• Comfortable working independently on well-defined problems, knowing when to ask for guidance, and communicating results clearly to technical and non-technical stakeholders.
• Будет плюсом: Nice to have: experience with feature flags and staged rollouts, containerised deployments (Docker/Kubernetes)

• Условий в вакансии нет