data engineer for data engineering automation

AgileEngine creates award-winning software for Fortune 500 brands and startups across more than 17 industries, with expertise in application development and AI/ML.

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL
• Build and operationalize agentic workflows to automate data engineering and operational processes, including data validation, issue identification, troubleshooting, and workflow execution
• Integrate agentic capabilities with existing Databricks, GCP, BigQuery, and Delta Lake environments
• Develop data pipelines and processing solutions to support new business requirements and datasets
• Build reusable frameworks and components for data engineering and business use cases
• Implement data quality checks, monitoring, validation, exception handling, and production controls
• Optimize PySpark and SQL workloads for performance, reliability, and scalability
• Support testing, deployment, productionization, and ongoing enhancement of data and agentic solutions
• Troubleshoot complex data and production issues and implement sustainable solutions
• Collaborate with business, data engineering, and platform teams to identify further automation opportunities

• 5+ Years of hands-on experience with Databricks and PySpark
• Advanced SQL and data-processing skills
• Hands-on experience with GCP, particularly BigQuery
• Experience with Delta Lake and modern data lake/lakehouse architectures
• Strong understanding of ETL/ELT, data pipeline design, performance optimization, and data quality
• Experience building reliable, scalable, production-grade data solutions
• Strong analytical and troubleshooting skills
• Understanding of software engineering practices, including testing, version control, deployment, monitoring, and production support
• Upper-intermediate English
• Будет плюсом: Experience developing or integrating AI/agentic workflows, AI agents, or workflow automation solutions; experience applying AI to automate data engineering, validation, troubleshooting, or operational processes; familiarity with orchestration and automation frameworks; experience developing reusable data engineering frameworks and platform components; exposure to productionizing AI-enabled solutions with appropriate validation, monitoring, and human oversight

• Competitive USD-based compensation
• Budgets for education, fitness, and team activities
• Mentorship, TechTalks, and personalized growth roadmaps
• Schedule flexibility, with the option to work from home or the office
• Projects with modern solutions and top-tier clients, including Fortune 500 enterprises and leading product brands