Azure data engineer

Описания нет

• Design, develop, and maintain complex data pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize large-scale data processing solutions using PySpark and Databricks.
• Collaborate with cross-functional teams to understand data requirements and deliver efficient data solutions.
• Implement CI/CD pipelines in Azure DevOps for automated testing, integration, and deployment of data solutions.
• Develop, test, and maintain robust data solutions in Python for data extraction, transformation, and loading (ETL) processes.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Ensure data security and compliance with company standards and best practices.
• Stay updated with the latest trends and best practices in cloud data engineering and Azure services.

• Experience with Azure Data Factory for orchestrating and automating data movement and transformation.
• Proficiency in Azure Databricks for building and managing scalable data processing frameworks.
• Strong programming skills in Python and experience working with PySpark for big data processing.
• Knowledge of Azure DevOps and experience with CI/CD pipeline implementation.
• Familiarity with SQL for querying and managing large datasets.
• Strong problem-solving skills and the ability to work independently or as part of a team.

• Условий в вакансии нет