Data engineer

A payment attempt, an order, a gift card, a sale in a store or online. At a large international retailer these start in different source systems, and this team turns them into one data product that analysts open in BigQuery without first having to find, combine and clean the data themselves. If a payment or an order is wrong in that product, everyone downstream is wrong with it.

• Build, test and run batch, streaming and near-real-time pipelines for sales, payment, order and gift-card data
• Write transformations and production data models in SQL, Python, BigQuery and dbt
• Reconciliation, data-quality checks, monitoring, and dead-letter queues for what fails
• Keep personal data safe through modelling and access controls
• CI/CD, code reviews, infrastructure automation, and documentation people can use
• Help guide design decisions and mentor colleagues

• Hands-on Databricks: Spark, Delta Lake, Unity Catalog, structured streaming, Workflows
• Strong Python and advanced SQL
• Pipelines and data models you built and then kept running in production
• Data-quality validation, reconciliation and alerting
• Event-driven architecture, change data capture, schema evolution, and how to recover when something breaks
• Git, automated testing, CI/CD
• Working knowledge of IAM, PII handling, metadata and lineage
• You live in the Netherlands and can work here without visa sponsorship
• Будет плюсом: BigQuery, Dataflow, Pub/Sub, Cloud Run, Cloud Storage; dbt, Apache Beam, Terraform, GitHub Actions; Sales, payments or other high-volume transactional data; Data mesh

• You'd be employed by BrightStone Group and work for the retailer. It isn't a job at the retailer itself. I'll tell you who they are on our first call
• Employment only. No freelancers or subcontractors, because the client won't accept them
• Travel is only for a team meeting, maybe once