Data platform engineer

грейд
senior
формат
офис Лиссабон
язык
Английский B1
опубликована
15 ноя

описание

At Chain IQ, your ideas move fast. Chain IQ is a global AI-powered leader in indirect procurement, headquartered in Baar, Switzerland, with operations across major delivery centers and 26 offices worldwide. We provide tailored, end-to-end procurement solutions that enable transformation, drive scalability, and deliver substantial reductions in our clients' indirect spend.

задачи

  • Develop and maintain modular ingestion patterns supporting batch, API-based, and event-driven data sources (e.g., JDBC, SaaS/ERP connectors, Kafka, CDC).
  • Establish reusable ingestion frameworks (metadata-driven when possible) that enable rapid and consistent onboarding of new data sources.
  • Build and optimize Databricks-based data pipelines (Spark, Delta Live Tables/Lakeflow) across the medallion architecture (Bronze, Silver, Gold).
  • Implement and evolve data governance capabilities, including access controls, lineage, cataloging, auditing, data classification, and privacy compliance.
  • Define and enforce data quality standards, data contracts, monitoring, and alerting across ingestion and transformation layers.
  • Implement data protection and privacy solutions, including masking, tokenization, pseudonymization, and other appropriate anonymization techniques to meet regulatory requirements (e.g. GDPR).
  • Build, own, and evolve data products aligned to a canonical data model, ensuring clear ownership, SLAs and documentation.
  • Collaborate with domain teams to integrate source data into the canonical model while preserving domain-specific nuance.
  • Champion a Data as a Product approach: discoverability, self-service, versioning, and consumer-facing contracts.
  • Build and maintain data consumption and serving layers, including BI/analytics datasets, APIs (REST/GraphQL), and use case-driven extensions such as streaming outputs or knowledge graph representations.
  • Drive technical standards and platform direction across CI/CD, Infrastructure as Code, testing, observability, and cost optimization
  • Mentor engineers and partner with governance, security, architecture, and business stakeholders.

требования

  • Around 6+ years in data engineering, with 2+ years designing/operating Databricks-based Lakehouse platforms at production scale.
  • Hands-on expertise with Apache Spark and Delta Lake; experience with Delta Live Tables/Lakeflow Declarative Pipelines is a strong plus.
  • Practical experience with Unity Catalog (or equivalent) for access control, lineage, and cataloguing.
  • Demonstrated experience implementing data anonymisation/pseudonymisation techniques in a regulated environment.
  • Solid understanding of canonical/conformed data modelling (e.g., data vault, dimensional modelling, domain-driven design for data).
  • Track record exposing data via APIs (REST/GraphQL) - designing schemas, contracts, and versioning for external/internal consumers.
  • Proficient in Python and/or Scala, SQL, and infrastructure-as-code (Terraform preferred).
  • Strong grasp of data governance frameworks, data privacy regulation (GDPR/CCPA), and data security practices.
  • Experience with cloud platforms (Azure, AWS, or GCP) and their native data/event services.
  • Будет плюсом: experience with Delta Live Tables/Lakeflow Declarative Pipelines is a strong plus

условия

  • Join a truly global team.
  • We offer a dynamic and international environment where high performance meets real purpose.
  • We're proud to be Great Place to Work-certified and even prouder of the people who make that possible.

навыки

компания

похожие вакансии

статьи для разработчиков