data engineer for scalable data pipelines

No description

• Design, develop, and maintain scalable data pipelines using Apache Spark, with Databricks preferred;
• Write efficient and optimized PySpark code for data transformation and processing;
• Develop and execute complex SQL queries for data extraction, validation, and reporting;
• Implement unit tests using pytest to ensure code reliability and maintainability;
• Collaborate with data scientists, analysts, and other engineers to deliver high-quality data solutions;
• Monitor and troubleshoot data workflows and performance issues;
• Document technical designs, processes, and best practices.

• 3+ Years of experience in Data Software Engineering;
• Proven experience with Apache Spark, ideally in a Databricks environment;
• Proficiency in PySpark and SQL;
• Background in unit testing frameworks, especially pytest;
• Understanding of data engineering principles and ETL processes;
• Familiarity with version control systems such as Git;
• Ability to work independently and in a collaborative team setting;
• Excellent problem-solving and communication skills;
• English proficiency at Upper-Intermediate level (B2) or higher;
• Nice to have: experience with cloud platforms such as Azure, AWS, or GCP, knowledge of CI/CD pipelines and DevOps practices, familiarity with Delta Lake, MLflow, or other Databricks-native tools.

• No conditions specified