AI Quality Architect (auto)

The role focuses on designing software quality systems where AI agents generate, execute, and maintain most of the testing. The architect defines how agents operate autonomously, how their results are verified, and where human oversight is required.

• Design the end-to-end AI-driven test architecture for generation, execution, self-healing, triage, and reporting
• Define the oracle strategy for specifying correctness so agents can verify it
• Build an evaluation framework for agents, including evals, regression sets, drift detection, and defect precision and recall
• Define autonomy levels, guardrails, and human-in-the-loop checkpoints; own AI-in-QA governance and risk
• Manage the context architecture exposed to agents, including specs, requirements, code, test knowledge, RAG, MCP tools, and knowledge graphs
• Control cost, latency, and flakiness in AI-based pipelines; integrate them into CI/CD
• Set standards and mentor QA engineers, analysts, and developers across teams

• Background in test architecture or quality engineering at scale, such as SDET, test automation architecture, or developer productivity
• Production experience building or operating LLM- or agent-based workflows; using only Copilot or ChatGPT is not sufficient
• Solid engineering skills in Python and/or TypeScript/Java, API and service architecture, CI/CD, and containers
• Understanding of LLM limitations, including hallucination, non-determinism, prompt and context design, and evaluation methods
• Experience designing test strategies for distributed systems, including microservices, event-driven, and API-first systems
• Ability to demonstrate measurable outcomes, such as escaped defects, cycle time, cost per test, and automation reliability
• Будет плюсом: experience testing AI/ML products, including LLM evals, red-teaming, safety and bias testing; MCP server or tool development; multi-agent orchestration; observability and test analytics; experience in telecom, billing, fintech, or enterprise B2B; public footprint through talks, articles, open-source, or patents

• Competitive salary
• Medical insurance covering dental services
• More than 300 hard- and soft-skills programs through the corporate career development center
• Open environment and knowledge-sharing culture
• Free English courses and opportunities to practice foreign languages daily
• Flexible working hours