описание
Описания нет
задачи
- Design, develop, and maintain complex data pipelines using Azure Data Factory and Azure Databricks.
- Build and optimize large-scale data processing solutions using PySpark and Databricks.
- Collaborate with cross-functional teams to understand data requirements and deliver efficient data solutions.
- Implement CI/CD pipelines in Azure DevOps for automated testing, integration, and deployment of data solutions.
- Develop, test, and maintain robust data solutions in Python for data extraction, transformation, and loading (ETL) processes.
- Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
- Ensure data security and compliance with company standards and best practices.
- Stay updated with the latest trends and best practices in cloud data engineering and Azure services.
требования
- Experience with Azure Data Factory for orchestrating and automating data movement and transformation.
- Proficiency in Azure Databricks for building and managing scalable data processing frameworks.
- Strong programming skills in Python and experience working with PySpark for big data processing
- Knowledge of Azure DevOps and experience with CI/CD pipeline implementation.
- Familiarity with SQL for querying and managing large datasets.
условия
- Условий в вакансии нет
навыки
компания
Лондонская компания IT-рекрутинга и консалтинга с 2000 года.
