Databricks Data Engineer

We are expanding our Data Engineering capability to support a growing client data platform built on modern cloud architecture. We are seeking a Databricks Data Engineer to accelerate the ingestion, processing, and transformation of data into a scalable analytics platform, delivering tangible business value at pace. This role will work closely with client stakeholders and internal teams to design, build, and optimise robust data pipelines while contributing to the evolution of an enterprise-grade data platform leveraging industry best practices.

• Design, build, and maintain scalable data pipelines using Databricks (Azure Databricks preferred)
• Develop and manage efficient data ingestion frameworks for onboarding diverse data sources
• Implement and optimise Delta Lake and Medallion (Bronze/Silver/Gold) architecture
• Collaborate with business, analytics, and reporting teams to structure data for optimal consumption
• Support the expansion and enhancement of the data platform, focusing on performance, scalability, and reliability
• Integrate and process data from multiple sources, including tools such as Qlik Replicate or similar CDC technologies
• Apply data modelling best practices to ensure high-quality, reusable datasets
• Monitor, troubleshoot, and optimise pipelines to ensure availability and performance
• Contribute to CI/CD, automation, and DevOps practices within data engineering workflows
• Work collaboratively within Agile delivery teams (Scrum / Kanban)

• Good understanding of:
  • Data engineering principles
  • Medallion Lakehouse architecture
  • Python / PySpark and SQL
  • Data analysis, reporting, and consumption layers (e.g. Power BI)
• Solid understanding of data modelling principles

• Familiarity with Qlik Replicate or similar data integration / CDC tools
• Knowledge of data governance and data quality frameworks
• Exposure to streaming data processing (e.g., Spark Structured Streaming)
• Soft Skills
  • Strong stakeholder engagement and communication skills
  • Ability to work effectively in collaborative, distributed teams
  • Proactive, delivery-focused mindset with a commitment to quality
  • Adaptable and able to ramp up quickly in new environments
  • Passion for continuous improvement and data engineering excellence

• Strong hands-on experience with Databricks, Spark, PySpark, and Delta Lake
• Proven experience building and maintaining ETL/ELT pipelines
• Experience working with modern cloud data platforms (Azure, AWS, or GCP)
• Experience working in Agile delivery environments
• Experience within Commercial Insurance / London Market / Financial Services
• Experience supporting analytics and reporting use cases

• Hybrid / Remote with travel to London 1–2 days per week as required

• Collaborative working environment – we stand shoulder to shoulder with our clients and our peers through good times and challenges 
• We empower all passionate technology loving professionals by allowing them to expand their skills and take part in inspiring projects 
• Expleo Academy - enables you to acquire and develop the right skills by delivering a suite of accredited training courses 
• Competitive company benefits
• Always working as one team, our people are not afraid to think big and challenge the status quo

• As a Disability Confident Committed Employer we have committed to:
• Ensure our recruitment process is inclusive and accessible
• Communicating and promoting vacancies
• Offering an interview to disabled people who meet the minimum criteria for the job
• Anticipating and providing reasonable adjustments as required
• Supporting any existing employee who acquires a disability or long term health condition, enabling them to stay in work at least one activity that will make a difference for disabled people

“We are an equal opportunities employer and welcome applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age”. 

We treat everyone fairly and equitably across the organisation, including providing any additional support and adjustments needed for everyone to thrive

#LI-BM1

#LI-DS1