AI QA (auto) for enterprise solutions

DataSpecta is a Data & AI consulting and engineering company helping organizations transform data into measurable business value through advanced analytics, modern data platforms and enterprise AI solutions. We design and deliver solutions across both on-premise and cloud environments, working closely with enterprise customers from discovery through production. Alongside customer projects, we build our own AI products such as CogniSpecta (enterprise knowledge & cognitive search) and CodeSpecta (AI-powered software engineering).

• Build versioned golden datasets and evaluation harnesses
• Gate production releases on evaluation baselines in CI
• Revalidate agents at each model version change
• Run quality control on recurring reports
• Run adversarial tests for prompt injection, data leakage and unsafe tool use
• Triage support and agent output incidents
• Maintain the audit evidence pack

• Based in Baku or willing to relocate
• 4+ years in QA or test engineering, including test automation ownership
• Proven experience evaluating LLM or ML outputs
• Strong Python skills for evaluation tooling
• Experience with formal defect management
• Fluent English
• Curiosity about solving business problems with AI
• Comfort across business, data and technology
• End-to-end ownership
• Continuous learning
• Будет плюсом: Audit evidence experience in a regulated sector; Model risk management frameworks

• Условий в вакансии нет