описание
Capitole is growing, and we're looking for a Senior Data Platform Engineer to join our team and help shape the future of a modern Azure-based Data Platform! We are looking for an experienced engineer who enjoys building, evolving and operating data platforms, with a strong focus on cloud infrastructure, orchestration, automation and governance. You will play a key role in the evolution of a Lakehouse architecture, working on platform foundations that enable data engineering teams to build and deliver reliable, scalable and efficient data products. This is a hands-on engineering role combining Data Engineering, Cloud Platform Engineering and DevOps.
задачи
- Own and evolve the Apache Airflow platform, including environments, deployments, DAG standards, monitoring and reliability.
- Contribute to the migration from Azure Data Factory (ADF) to Apache Airflow.
- Develop metadata-driven orchestration frameworks, reusable operators and automated DAG generation.
- Manage Azure resources using Terraform and Infrastructure as Code.
- Work with Azure Key Vault, Managed Identities, Service Principals, Entra ID and RBAC.
- Manage CI/CD pipelines and multi-environment deployments using Azure DevOps.
- Operate and optimize Azure Databricks, Unity Catalog, Delta Lake and Data Lake Storage.
- Improve platform observability, monitoring, alerting, performance and cost efficiency.
- Contribute to Data Catalog, Data Quality and data lineage initiatives.
- Participate in architecture decisions, technical reviews and engineering standards.
- Collaborate with Data Engineers, Architects and Product Owners to continuously improve the platform.
требования
- 6+ years of experience in Data Engineering, Data Platform Engineering or Cloud Engineering.
- Strong knowledge of Python, SQL and PySpark.
- Hands-on experience with Apache Airflow in production environments.
- Azure Data Factory and Azure Data Lake Storage (ADLS).
- Azure Key Vault and identity management.
- Azure RBAC and Entra ID.
- Azure DevOps and CI/CD pipelines.
- Strong experience with Terraform and Infrastructure as Code.
- Hands-on experience with Azure Databricks, Delta Lake and Unity Catalog.
- Knowledge of Lakehouse architecture, Medallion architecture and metadata-driven frameworks.
- Experience with monitoring, observability and production troubleshooting.
- Good communication skills and the ability to collaborate with different technical teams.
- English communication skills.
- Будет плюсом: Experience migrating orchestration workloads from ADF to Airflow; Experience building reusable orchestration frameworks and metadata-driven DAGs; Airflow deployments on Kubernetes or experience with AKS; Knowledge of FinOps practices and Azure/Databricks cost optimization; Experience with AI-assisted engineering workflows, MCP, AI agents or automated code reviews
условия
- Permanent contract from day one.
- €1,200 annual budget for professional development and certifications.
- Private health insurance fully covered by Capitole.
- Hybrid working model in Barcelona.
- Monthly feedback and continuous career development.
- Wellhub membership and Payflow benefits.
- Tech communities, team events and an international environment.
- At Capitole, we believe in people, technical excellence and building strong teams. You'll have the opportunity to work on challenging projects, influence technical decisions and grow alongside talented professionals.