automation qa for agentic AI systems

EPAM’s enterprise-grade Agent Development Platform is a production-grade, cloud-native ecosystem that enables engineering teams to define, orchestrate, deploy, and observe AI agents at scale. The platform standardizes agent development using LangGraph and Strands Agents on AWS AgentCore Runtime.

• Own security, functional, and performance test suites for the A2A gateway;
• Validate Cedar policy enforcement correctness across LOG_ONLY and ENFORCE modes;
• Conduct trust model gap analysis for non-AgentCore A2A agents;
• Design and execute functional and security test strategies for agentic AI systems;
• Build and maintain Python-based security test automation frameworks;
• Perform performance testing and benchmarking for cloud APIs using k6 and Locust;
• Test permit/deny correctness and forbid-overrides-permit logic within Cedar policies;
• Apply agentic AI and LLM threat modeling practices to identify risks such as excessive agency and tool parameter exfiltration;
• Evaluate A2A trust model components, including OAuth 2.0, signed Agent Cards, JWT validation, and token scope enforcement for agent channels.

• 3+ Years of experience in security engineering or QA;
• Expertise in API security testing and performance benchmarking for cloud APIs;
• Skills in security test design for AI/agent systems beyond traditional REST APIs;
• Background in enforcement mechanism design or implementation;
• Knowledge of A2A trust model concepts, including OAuth 2.0, signed Agent Cards, JWT validation, and token scope enforcement;
• Proficiency in Python security test automation, functional test design, and performance testing tools such as k6 and Locust;
• Understanding of Cedar policy testing, including permit/deny correctness and LOG_ONLY vs ENFORCE modes;
• Familiarity with agentic AI and LLM threat modeling frameworks, including OWASP Top 10 for LLMs;
• English proficiency at B2 level or higher;
• Nice to have: Familiarity with multi-agent communication security patterns, expertise in trust model gap analysis for non-AgentCore A2A agents.

• Hybrid work in Kazakhstan and 4 other locations.