описание
The SaaS Data Layer implementation team builds and maintains client environments for a new trading platform. Its applications primarily consist of data pipelines that transform data for downstream systems or ingest and transform data from upstream systems.
задачи
- Develop, implement, and maintain data pipelines and ETL processes using Azure Databricks
- Design and manage Azure Databricks for optimized data storage and retrieval
- Collaborate with cross-functional teams to define and implement data integration solutions across multiple sources and platforms
- Implement data governance, security, and compliance best practices in cloud-based solutions
- Monitor and troubleshoot performance issues in Azure-based applications and data pipelines
- Automate deployment and monitoring processes using CI/CD pipelines
- Stay updated on emerging Azure technologies and recommend improvements
- Produce architectural design diagrams and technical documentation
требования
- 10+ Years of development experience
- 3+ Years of hands-on experience with Azure cloud services, including Databricks and DevOps
- Strong proficiency in Python and PySpark or SQL for data transformation and analysis
- Experience with real-time data streaming and event-driven architectures
- Deep understanding of ETL/ELT workflows and big data processing frameworks
- Knowledge of Azure DevOps and CI/CD pipelines for automated deployments
- Familiarity with data security, governance, and compliance best practices in cloud environments
- Excellent problem-solving skills and ability to work in an agile development environment
- Strong communication and collaboration skills
- Будет плюсом: Azure network concepts, Azure Service Bus, Azure Functions, Agile, Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect certifications, Power BI or other visualization tools, machine learning and AI-based analytics on Azure
условия
- Professional training and career growth opportunities
- Remote work from Poland
навыки
компания
Поставщик инженерных ИТ-услуг и цифровой трансформации (Luxoft, DXC).