описание
We are looking for a Data Engineer Analyst with at least 2 years of experience in Data Engineering or Big Data environments to join a data-driven project focused on the development and evolution of a DataLake and Limits solution. The ideal candidate combines strong hands-on experience with Python, SQL, ETL development and data pipelines with the analytical ability to understand complex data models, transformations, dependencies, and data quality requirements.
задачи
- Analyze and understand existing data models, data flows and transformation processes.
- Understand existing datasets, mappings and dependencies within the DataLake.
- Develop, maintain and optimize Python processes, ETLs and data pipelines.
- Build and maintain Control-M jobs and batch processing workflows.
- Implement data ingestion, transformation and processing logic.
- Develop and adapt data models and datasets required for the Limits solution.
- Perform SQL-based data analysis, validation and reconciliation across multiple data sources.
- Implement data quality controls and investigate and resolve data inconsistencies.
- Support testing, certification, deployment and production releases.
- Create and maintain technical documentation covering data models, processes, mappings and transformations.
- Collaborate with technical and functional teams to ensure data solutions meet business and technical requirements
требования
- 2+ years of professional experience in Data Engineering, Big Data or similar environments.
- Solid experience with Big Data / DataLake architectures.
- Strong SQL skills for querying, analysis and data validation.
- Hands-on experience with Python.
- Experience developing ETL processes.
- Knowledge of data ingestion and transformation processes.
- Experience developing and maintaining data pipelines.
- Good understanding of data modeling concepts.
- Experience with Control-M and/or batch processing.
- Strong skills in data analysis, validation and reconciliation.
- English level B2 or above
- Будет плюсом: Atoti / ActiveViam; Databricks; REST APIs and Microservices; Jira / Confluence; Financial services or banking environments; Counterparty Risk / Limits concepts
условия
- B2B or employment contract
- Remote within CET Time zone.
- Possibility to be part of a multicultural team and work on international projects.
