Lead Data Engineer

                                                                            Job Description – Lead Data Engineer

Data Engineer (Azure | Databricks | PySpark)
Permanent | Hybrid | UK Public Sector Project

OVERViEW

We are seeking a hands-on Data Engineer to join a growing Data & Analytics practice. This role focuses on building, optimising, and maintaining modern cloud data platforms.

You will work across the full data lifecycle, delivering scalable, high-performance solutions using Azure and Databricks.

Who We’re Looking For

A strong Data Engineer with deep technical expertise in Azure and Databricks, who enjoys building robust pipelines and working in a collaborative, client-facing environment.

Key Responsibilities

• Build and maintain scalable data pipelines (batch & real-time)
• Develop solutions using Azure Databricks and PySpark
• Implement Medallion Architecture (Bronze, Silver, Gold)
• Design ETL/ELT pipelines using Azure Data Factory
• Develop data warehouse and lakehouse solutions
• Support BI, analytics, and AI workloads
• Optimise performance, cost, and reliability
• Implement CI/CD pipelines using Azure DevOps
• Collaborate with architects, analysts, and stakeholders

Mandatory Requirements

• Strong hands-on experience with Azure Databricks (PySpark)
• Experience with Azure Data Factory and Synapse
• Strong SQL and data warehousing expertise
• Experience with data modelling (star, snowflake, Kimball)
• Experience with Power BI and Microsoft Fabric
• Strong programming skills (Python, SQL, scripting)
• Experience with CI/CD and DevOps practices
• Ability to produce technical documentation (HLD, LLD)

Desirable

• Experience in consulting / professional services
• Exposure to AI and advanced analytics solutions
• Experience with real-time / streaming data
• Experience in regulated industries

Soft Skills

• Strong problem-solving skills
• Collaborative and team-oriented mindset
• Clear communication and documentation skills
• Ability to work in fast-paced delivery environments

Contract Details

• Role Type: Permanent & Hybrid (2 days per week in office)
• Location: Guildford
• Clearance: Candidates must be eligible for UK Security Clearance (SC) or active SC