Data engineer in local government

This is an opportunity to play a leading role in developing Barnet’s modern data capability and helping the council gain greater value from its information. As a Data Engineer in our Insight & Intelligence Hub, you will build, connect and improve the data pipelines, platforms and products that support better decisions, more efficient services and improved outcomes for residents.

• You will help maintain our Microsoft Azure Data Platform and support its transition to Microsoft Fabric, integrating data from operational, supplier-hosted and cloud-based systems.
• You will design robust ETL processes, APIs and reusable data workflows; improve data quality, reliability and accessibility; and prepare trusted datasets for analytics, AI and machine learning.
• Working across the council, you will translate user needs into secure and scalable technical solutions while ensuring compliance with UK GDPR, information governance and security requirements.
• You will also lead complex data engineering projects, manage relationships with suppliers and stakeholders, and help establish strong standards and practices for data engineering across Barnet.

• You will be an experienced Data Engineer with strong applied knowledge of modern data engineering technologies, cloud platforms and production data services.
• You will be educated to degree level, have an equivalent qualification, or demonstrate relevant professional experience.
• Strong experience of cloud data engineering platforms and tools, such as Microsoft Azure, Data Factory, Synapse, Spark, Snowflake or Databricks.
• Experience of designing and maintaining secure data pipelines, ETL processes and lakehouse architectures.
• Proficiency in Python and SQL, together with experience of relational and NoSQL databases.
• Experience of integrating, cleaning, modelling and scaling complex datasets from operational systems.
• Knowledge of version control, automated testing, metadata management and production support.
• Experience of leading data projects and delivering reliable data products or services.
• A sound understanding of UK GDPR, privacy by design, information security and the responsible handling of sensitive data.
• The ability to explain complex technical issues clearly to non-technical colleagues and influence stakeholders at different levels.
• Strong problem-solving, prioritisation and relationship-building skills, with a commitment to continuous improvement and collaborative working.
• You will bring curiosity, personal drive and an outcomes-focused approach, alongside a commitment to inclusion, professional development and sharing knowledge with others.
• Будет плюсом: Experience of Microsoft Fabric, event streaming, machine-learning datasets, Power BI or other visualisation tools would be an advantage

• 31 days annual leave, plus public and bank holidays
• Access to the Local Government Pension Scheme , which provides a valuable guaranteed income in your retirement together with security for your dependents
• A comprehensive Flexible Working Policy which supports work-life balance by empowering staff and management to make decisions around when, where, and how work is done
• A vast range of lifestyle discounts from major retailers, supermarkets, energy suppliers and more
• Broad range of payroll benefits including cycle to work, eye care vouchers, travel and gym membership
• Excellent training and development opportunities
• Employee well- being training programs including confidential employee assistance