Data engineer in AI

грейд
senior
формат
гибрид Лиссабон
язык
Английский C1
опубликована
вчера

описание

Join an international organization building a regional AI Innovation Capability Hub from Lisbon. You will become part of an environment focused on turning Generative AI opportunities into production-ready solutions that can ultimately scale across a wider international business. The Senior Data Engineer will be responsible for designing, developing, and optimizing scalable Data pipelines and architectures within a modern Data platform leveraging Azure and Databricks, ensuring that Data is accessible, reliable, and actionable.

задачи

  • Design, develop, and optimize scalable Data pipelines, Data lakes, and Delta tables (using Azure Data Factory, Databricks, Apache Spark, Delta Lake, and Azure Data Lake Storage Gen2), ensuring performance, scalability, reliability, and cost efficiency;
  • Implement Data Quality, Governance, and Security best practices across Data pipelines and architectures;
  • Monitor and troubleshoot Data pipelines and production jobs (using Azure Monitor and Databricks job runs);
  • Collaborate with Data Scientists, BI Developers, Analysts, and Business Stakeholders to translate Business requirements into Data Solutions and enable self-service analytics;
  • Integrate Data Solutions with Power BI, Power Apps, Microsoft 365 tools, and other enterprise platforms;
  • Lead technical workstreams and contribute to reusable pipeline components, development standards, architecture best practices, and peer code reviews;
  • Provide mentorship and technical guidance to junior Data Engineers and Analysts, promoting best practices across coding, architecture, and DevOps.

требования

  • Bachelor’s or Master’s Degree in Computer Science, Engineering, or a related field;
  • + 5 years of professional experience in Data Engineering, or related roles;
  • Proven experience designing, developing, and optimizing scalable Data pipelines and distributed Data Solutions (using Databricks, Apache Spark (PySpark or Scala), and Delta Lake);
  • Experience working with Azure Cloud Services (including Azure Data Factory and Azure Data Lake Storage Gen2);
  • Experience programming with SQL and Python, and working with distributed Data processing;
  • Experience with CI/CD, Infrastructure as Code, and DevOps practices (using tools such as Terraform, GitHub Actions, or Azure DevOps);
  • Experience implementing Data Quality, Governance, Security, Monitoring, and reliability practices across Data pipelines;
  • Experience collaborating with technical and non-technical Stakeholders in Global environments, as well as leading technical workstreams and mentoring Junior team members;
  • Knowledge of Data Modeling techniques and modern Data engineering practices;
  • Excellent communication skills – Fluency in English.
  • Будет плюсом: Azure Certifications (such as Azure Data Engineer Associate, or related certifications – would be a plus); Familiarity with Data Governance frameworks and tools (such as Azure Purview, Unity Catalog, Data lineage, Data cataloging, and metaData management), as well as real-time Data streaming technologies (such as Kafka or Azure Event Hubs); Familiarity with Machine Learning workflows, MLOps on Databricks, Data privacy regulations (such as GDPR or HIPAA), and Agile and Waterfall methodologies (would be a plus)

условия

  • Условий в вакансии нет

навыки

компания

похожие вакансии

статьи для разработчиков