platform engineer for AI data platforms

Goldman Sachs is a global investment banking, securities and investment management firm. The Lakehouse and AI Data Platform team builds data foundations that support the firm’s AI and analytics capabilities.

• Build, enhance and support batch and streaming data pipelines on the Lakehouse and AI data platform;
• Refactor and modernise existing data flows to improve reliability, performance and maintainability;
• Build reusable tooling to improve delivery, consistency and operational support;
• Ensure data pipelines are production-ready, well tested and operationally supportable;
• Develop raw, refined and curated datasets for analytics, reporting and AI use cases;
• Apply data modelling principles to represent business entities, relationships and historical change;
• Work with consumers to shape usable, documented data products aligned with business needs;
• Implement controls to validate data completeness, accuracy and consistency;
• Use reconciliation approaches to validate production outputs and investigate data breaks;
• Contribute to standards for testing, monitoring and issue resolution;
• Improve testing, monitoring and reconciliation tooling to strengthen platform reliability and delivery;
• Work with engineers, platform teams and data consumers to deliver agreed outcomes on time and to quality expectations;
• Communicate progress, risks, dependencies and design choices;
• For more senior candidates, contribute to technical leadership, task breakdown and support for junior engineers.

• Bachelor’s or master’s degree in a relevant discipline, or equivalent practical experience;
• Strong quantitative skills or data engineering expertise;
• Strong hands-on programming experience in Python or Java;
• Good working knowledge of SQL, including troubleshooting, optimisation and data analysis;
• Ability to learn new tools, internal platforms and delivery workflows quickly;
• Familiarity with version control, testing, release discipline and CI/CD practices;
• Understanding of temporal data modelling, schema design, schema evolution and data compatibility;
• Understanding of partitioning, clustering and other techniques for improving data performance at scale;
• Ability to choose between normalised and denormalised models and between natural and surrogate keys;
• Practical approach to data quality, reconciliation and root-cause analysis;
• Experience building or supporting production data pipelines in a collaborative engineering environment;
• Experience with distributed data processing frameworks such as Apache Spark;
• Working knowledge of JSON, Avro and Parquet;
• Nice to have: technical design ownership across multiple datasets or pipeline domains, experience guiding implementation standards and engineering practices, ability to lead delivery for a workstream and support less experienced engineers.

• The role is based in London, England, United Kingdom;
• Training and development opportunities, firmwide networks, benefits, wellness and personal finance offerings, and mindfulness programs are available;
• Reasonable accommodations are available for candidates with special needs or disabilities during the recruiting process.