Data Engineer

Graphcore is a developer of hardware, software, and systems infrastructure designed to advance artificial intelligence compute and support the adoption of AI solutions across various industries. As a member of the SoftBank Group, the organization focuses on enabling artificial super intelligence and creating accessible, transformative technologies through a multidisciplinary approach involving AI research, silicon design, and systems architecture.

• Lead the design, build, and evolution of robust data pipelines and platform services for analytics, reporting, and operational use cases;
• Own the data engineering stack, planning and delivering improvements to reliability, scalability, maintainability, performance, and security;
• Build and operate Python-based batch and streaming workflows, including orchestration, testing, deployment, monitoring, and incident resolution;
• Design and implement secure, resilient, and cost-conscious data solutions on AWS using services such as S3, Lambda, Aurora PostgreSQL, Athena, Glue, and Redshift;
• Define and apply engineering standards for data quality, observability, documentation, release processes, and operational support;
• Partner with analysts, engineers, and business stakeholders to translate requirements into trusted datasets, data models, and reusable data products;
• Drive improvements to platform resilience through idempotent processing, retry and recovery mechanisms, buffering strategies, and backfill or replay capabilities;
• Lead technical decision-making by reviewing designs and code, sharing expertise, and raising the quality bar for data engineering;
• Build and maintain CI/CD workflows and development practices for efficient delivery of data infrastructure;
• Ensure appropriate data protection and access controls, including least-privilege access, secure secrets handling, and database permissions;
• Contribute to the development of internal tools and lightweight applications to support self-serve workflows;
• Identify opportunities for platform and process improvements to shape the direction of data engineering.

• Strong experience designing, building, and operating production-grade data pipelines and platforms in Python;
• Strong hands-on experience with modern data orchestration, testing, deployment, and monitoring practices;
• Experience building solutions on AWS data services including storage, processing, and query technologies;
• Strong understanding of data modelling, data quality, schema design, and performance optimisation across relational and analytical systems;
• Experience designing reliable data systems that recover from failure and operate effectively in production;
• Experience working with batch and streaming data pipelines, including operational support and troubleshooting;
• Strong knowledge of security and access control principles, including IAM, database permissions, and secure handling of credentials;
• Experience providing technical leadership as a senior individual contributor through design reviews, code reviews, standards-setting, and mentoring;
• Ability to work effectively with technical and non-technical stakeholders to turn business needs into scalable solutions;
• Strong communication skills with the ability to explain technical decisions and influence outcomes;
• Nice to have: Prefect, streaming technologies, PostgreSQL, Redshift, ClickHouse, CI/CD tooling, Infrastructure as Code, Streamlit, Flask, dbt, cloud cost optimisation, experience in fast-moving engineering-led environments.

• No conditions specified