flask developer for AI coding evaluation

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems.

• Build realistic developer environments with codebases, infrastructure, tickets, documentation, conversations, and believable development histories;
• Design tasks from intermediate states of simulated environments;
• Craft prompts and define what constitutes a solved task;
• Ensure tasks are solvable by an AI agent;
• Write functional and integration tests that verify agent solutions;
• Accept valid approaches and reject incorrect ones without making tests too strict or too lenient;
• Iterate on tasks and tests based on QA feedback;
• Review agent solutions and analyze failures;
• Refine tasks and tests until evaluations are fair and robust.

• 8+ Years in software development;
• Strong experience with Python and FastAPI;
• Experience with JavaScript/TypeScript and React;
• Experience with Docker, Postgres, Kafka, and Redis;
• Experience writing functional and integration tests;
• English proficiency at B2+.

• Part-time, project-based participation rather than permanent employment;
• Tasks are estimated to take 30 hours to complete depending on complexity; this is not a schedule requirement, and you choose when and how to work;
• Tasks must be submitted by the deadline and meet the listed acceptance criteria;
• Paid per accepted task;
• Rate depends on the qualification tier and task completion efficiency;
• Compensation is up to the equivalent of $150/hr.