описание
We are seeking a Principal Data Engineer to serve as one of the most senior data engineers on our team and a near-expert practitioner in the domain. This role combines deep technical execution with broad influence: tackling the most complex data engineering problems, designing and implementing data architectures, and raising the bar for engineering excellence across the Data Organization.
задачи
- Take on the most technically demanding data engineering work – high-scale pipelines, performance-critical workloads, workload optimization, and platform-level capabilities on Databricks and Azure – where deep expertise is essential to success.
- Write, review, and refactor production code that exemplifies the team’s standards for performance, reliability, security, observability, and cost efficiency.
- Help define and continuously evolve the engineering and architectural principles, patterns, and reference implementations used across the Data Organization.
- Maintain near-expert depth in Databricks and Azure data services and proactively build expertise in adjacent and emerging technologies on our roadmap.
- Design end-to-end data architectures – covering ingestion, storage, transformation, serving, and governance – using Lakehouse patterns on Databricks and the Azure data ecosystem.
- Implement and validate critical components of the architectures you design, ensuring they are demonstrably production ready.
- Partner with the Architecture function to ensure designs align with enterprise standards for security, governance, scalability, and total cost of ownership.
- Mentor data engineers across all levels through code reviews, pairing, design reviews, and direct coaching, with a particular focus on accelerating mid-level engineers toward senior contribution.
- Lead internal tech talks and written deep dives on patterns, pitfalls, and platform capabilities to multiply the team’s expertise.
- Embed into Data Engineering teams for engagements ranging from days to months to lead, accelerate, or de-risk critical projects, transferring expertise back to the host team upon exit.
- Proactively identify technical risks, design flaws, and execution gaps, and drive issues to resolution with clear, well-reasoned recommendations.
- Serve as the go-to technical voice for complex data problems and platform capabilities.
- Translate business needs into clear technical strategies and translate technical trade-offs into language that supports informed decisions.
- Collaborate with peers in Architecture, Platform, AI, Analytics, Security, and Governance to ensure data engineering work integrates cleanly into the broader data and technology landscape.
требования
- Extensive data engineering experience at increasing levels of seniority, with a clear track record of delivering production-grade data platforms and pipelines at scale.
- Near-expert hands-on proficiency with Databricks (Spark, Lakeflow, Spark Declarative Pipelines (DLT), Delta Lake, Lakebase/Postgres, Unity Catalog, etc.) and the Azure data ecosystem.
- Demonstratable experience designing and implementing end-to-end data architectures for complex, enterprise-scale environments.
- Proven ability to mentor engineers, lead through influence (without direct reports), and raise team-wide standards for technical excellence.
- Familiarity with modern data architecture patterns (Lakehouse, medallion, data mesh), DataOps practices, and metadata-driven and configuration-driven pipeline frameworks and a strong instinct for reusable, scalable engineering patterns.
- Будет плюсом: Experience embedding across multiple teams, geographies and time zones as a senior technical contributor or technical lead; Familiarity with CI/CD and infrastructure-as-code tooling for data pipelines using Azure DevOps, Databricks Automation Bundles (DABS), GitHub Actions, or equivalent; Good understanding of data governance, metadata management, and cataloguing in enterprise environments
условия
- Условий в вакансии нет