AI software test engineer (auto) in API-first systems

грейд
senior
формат
гибрид Лиссабон
язык
Английский C1
опубликована
сегодня

описание

Join an international organization building a regional AI Innovation Capability Hub from Lisbon. You will become part of an environment focused on turning Generative AI opportunities into production-ready solutions that can ultimately scale across a wider international business.

задачи

  • Test AI/ML and rule-based systems for accuracy, consistency, and reliability, applying pattern-based, range-based, and semantic validation techniques;
  • Perform prompt-based testing (including edge cases and adversarial inputs) and validate AI systems (using tools such as Promptfoo, DeepEval, Ragas, Garak, and Giskard);
  • Apply Responsible AI and LLM evaluation practices (including bias, safety, and hallucination testing);
  • Design, execute, and maintain manual and automated test cases for APIs, web applications, and AI components, covering regression, system, and performance testing prior to production releases;
  • Perform API and Back-End testing (using Postman and Newman), validating JSON responses, API contracts, and schema compliance;
  • Develop and maintain test automation (using Playwright with JavaScript, TypeScript, or Python, and Selenium), leveraging AI-assisted tools (such as GitHub, Copilot, and ChatGPT, where appropriate);
  • Perform SQL-based validation and ensure Data consistency across API, processing, and output layers;
  • Integrate and execute automated tests within CI/CD pipelines (using tools such as Azure DevOps, Jenkins, and GitHub Actions), supporting regression and release validation;
  • Identify, document, and track defects (using Jira and TestRail), while contributing to debugging and root cause analysis;
  • Collaborate closely with Developers, Product Owners, and AI Engineers throughout Agile sprints, contributing to sprint planning, testing activities, and User Acceptance Testing.
  • A typical day will include testing APIs, applications, and AI-enabled systems, validating results, maintaining automated tests, supporting releases, investigating defects, and working with cross-functional teams to ensure quality and reliability.

требования

  • Bachelor’s Degree in Computer Science, Software Engineering, or a related field;
  • + 3 years of professional experience in Quality Assurance or Software Testing;
  • Experience in API and Back-End testing (including hands-on experience with Postman and Newman);
  • Experience designing and maintaining automated tests (such as Playwright and/or Selenium);
  • Experience working with CI/CD environments and tools (such as Azure DevOps, Jenkins, and/or GitHub Actions);
  • Experience using defect and test management tools (such as Jira and TestRail);
  • Exposure to AI systems and experience in testing AI/ML and rule-based solutions;
  • Advanced knowledge of SQL and experience performing Data validation;
  • Programming knowledge in Python and/or JavaScript;
  • Knowledge of AI and LLM testing and evaluation tools (such as Promptfoo, DeepEval, Ragas, Giskard, and Garak);
  • Understanding of Responsible AI and model evaluation principles (including bias, safety, and hallucination testing);
  • Familiarity with Azure AI Foundry and Azure OpenAI, as well as awareness of LangChain and LangGraph;
  • Excellent communication skills –
  • Fluency in English.
  • Будет плюсом: Certifications in ISTQB, Selenium, Cypress, or related testing frameworks (would be a plus); Knowledge of performance testing (such as JMeter or Locust), as well as an understanding of RAG pipelines and vector Database fundamentals (would be a plus)

условия

  • Have the opportunity to join a cross-functional team in an international company with a multicultural working environment!

навыки

компания

похожие вакансии

статьи для QA-инженеров