data engineer for ML modeling teams

EPAM builds enterprise software products, open source solutions, and accelerators, including data engineering solutions for client stakeholders.

• Support a team of data engineers in building pipelines used by MLOps and ML Engineers on ML modeling teams;
• Develop, optimize, and maintain data transformation pipelines in Databricks using PySpark;
• Work with data stored in ADLS Gen2 and SAP HANA Data Lake, primarily in Delta and Parquet formats;
• Implement and maintain data quality checks, including schema validation, deduplication, enrichment, and tagging;
• Communicate with stakeholders to understand business processes and model input data;
• Tune performance for large-scale datasets.

• 3+ Years of experience in data engineering or a related field;
• Proficiency in Python, PySpark, and Databricks, including Delta Lake;
• Familiarity with software version control tools such as GitHub and Git;
• Experience with CI/CD frameworks such as GitHub Actions;
• Knowledge of data lake technologies;
• Experience working with MS Azure;
• English proficiency at B2 level or higher;
• Nice to have: familiarity with at least one other programming or scripting language, such as Java, SQL, or Scala.

• Relocation support is available.