Data Engineer

EPAM develops enterprise software products, open source solutions, and technology accelerators.

• Design, develop, test, and maintain Python-based data pipelines;
• Write high-quality, maintainable Python code to migrate data between different storage systems with modifications;
• Create ETL jobs, Airflow operators, and Spark jobs to enhance data processing capabilities;
• Implement Data Intake processes for data sources, including ETL/ELT, DWH modelling, and optimization;
• Perform data cleansing, transformation, and quality assurance to maintain data integrity;
• Optimize data processing, storage, and retrieval to improve performance;
• Collaborate closely with cross-functional teams to ensure data accuracy and consistency.

• Write maintainable and scalable Python code;
• Strong understanding of distributed computation and multiprocessing;
• Knowledge of data warehouse concepts and ETL processes;
• Proficiency with state machines, web back-end development, networking, and protocols;
• Experience with cloud computing platforms such as AWS or GCP, and tools like Step Functions and Batch;
• Knowledge of Spark, Hive, Databricks, and other data processing frameworks;
• Familiarity with data orchestration tools like Airflow;
• Expertise in SQL, NoSQL, and cloud-based data storage solutions.

• Office in Kyrgyzstan.