Lead Data Engineer

Lead Data Engineer (Azure | Databricks | Fabric | PySpark | SQL)

Permanent | Hybrid | UK Public Sector Project

Overview

We are seeking a hands-on Lead Data Engineer to join a growing Data & Analytics practice. This role focuses on designing, building, optimising, and maintaining enterprise-level data solutions.

You will work across the full data lifecycle, delivering scalable, high-performance solutions using Azure and tools such as Databricks and Fabric.

Who We’re Looking For

A strong Data Engineer with deep technical expertise in Azure, Databricks and Fabric, who enjoys working in a collaborative, client-facing environment and thrives at designing and delivering enterprise-level data solutions, spanning initial requirements gathering, proof-of-concept development, architecture planning, deployment and delivery.

Key Responsibilities

· Develop solutions using Azure Databricks and/or Fabric

· Implement Medallion Architecture (Bronze, Silver, Gold)

· Develop data warehouse and lakehouse solutions

· Support BI, analytics, and AI workloads

· Optimise performance, cost, and reliability

· Collaborate with technical and functional stakeholders

Mandatory Requirements

· Proven ability to make both high-level strategic and day-to-day operational decisions with confidence

· Strong leadership and interpersonal skills, with a track record of motivating and influencing stakeholders at all levels

· Sharp analytical mindset with the ability to proactively spot emerging opportunities and resolve challenges before they escalate

· Strong hands-on experience with Azure Databricks and Fabric

· Strong SQL and data warehousing expertise

· Experience with data modelling

· Experience with Databricks AI/BI or Power BI

· Experience with CI/CD and DevOps practices

Desirable

· Experience in consulting / professional services

· Exposure to AI and advanced analytics solutions

· Exposure to Agentic AI systems built on top of Large Language Models

· Experience with real-time / streaming data

· Experience with Infrastructure as Code (Terraform or Bicep)

· Experience in regulated industries

· Passionate about staying at the forefront of industry developments, with a habit of actively monitoring emerging trends, tools, and technologies

· Databricks/Microsoft Azure certifications

Soft Skills

· Strong problem-solving skills

· Collaborative and team-oriented mindset

· Clear communication and documentation skills

· Ability to work in fast-paced delivery environments

Contract Details

· Role Type: Permanent & Hybrid (2 days per week)

· Location: Guildford

· Clearance: Candidates must be eligible for UK Security Clearance (SC) or active SC