Data Engineer / Data Architect

Role 1:

Role Title: Data Engineer
Location: London, UK
Days on site: 2-3 days per week

Role Description:

A Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and data platforms that enable reliable data ingestion, transformation, storage, and consumption across the organization. The role focuses on ensuring data availability, quality, security, and performance to support analytics, reporting, AI/ML, and business decision-making initiatives. Data Engineers collaborate closely with architects, analysts, and business stakeholders to deliver robust and efficient data solutions.

**Key Responsibilities:**

- Design, develop, and maintain end-to-end data pipelines and ETL/ELT processes.
- Build and optimize data warehouses, data lakes, and cloud-based data platforms.
- Integrate data from multiple internal and external sources.
- Ensure data quality, integrity, security, and governance standards are maintained.
- Monitor and troubleshoot data pipelines to ensure reliability and performance.
- Collaborate with Data Architects, Data Analysts, and business teams to support data and analytics requirements.
- Implement data modeling, transformation, and performance optimization best practices.
- Support advanced analytics, reporting, and AI/ML initiatives through scalable data engineering solutions.

**Key Skills:** Python, SQL, ETL/ELT, Data Warehousing, Data Lakes, Spark, Hadoop, Azure Data Factory, Databricks, Snowflake, Kafka, Cloud Platforms (Azure/AWS/GCP), Data Modeling, Data Governance, Performance Optimization, and CI/CD.

Role 2:

Role Title: Data Architect – AWS Databricks Migration & Data Modernization
Location: London, UK
Days on site: 2-3

Role Description:

We are seeking an experienced Data Architect to lead the assessment, modernization, and migration of a legacy enterprise Data Warehouse (DWH) platform to a modern AWS and Databricks ecosystem. The role will be responsible for conducting a comprehensive current-state analysis of the existing SQL Server-based data warehouse landscape, leveraging AI-enabled tools to accelerate discovery, lineage mapping, and migration planning.

The successful candidate will define the future-state architecture, data model, migration roadmap, and data product strategy while ensuring seamless transition of downstream consumers, reporting platforms, and business functions onto the new cloud-native environment.

**Key Responsibilities**

- Analyze the existing SQL Server Data Warehouse environment, including 2,000+ SQL scripts, stored procedures, ETL processes, and reporting dependencies.
- Utilize AI-powered platforms such as Claude, Devin, and other automation tools to accelerate code analysis, lineage discovery, dependency mapping, and migration planning.
- Develop end-to-end data lineage across source systems, transformation layers, data assets, reports, and business domains.
- Assess data consumption patterns, business personas, and reporting usage to identify dormant datasets, redundant processes, and rationalization opportunities.
- Document source-to-target mappings, cross-domain dependencies, and upstream/downstream impacts.
- Define the target-state data architecture on AWS and Databricks, including data models, storage patterns, integration architecture, and governance frameworks.
- Design the end-to-end migration strategy, covering data, ETL, reporting, infrastructure, security, and operational considerations.
- Create detailed migration roadmaps, wave plans, transition strategies, and implementation approaches for phased delivery.
- Define future-state data products and domain-aligned architectures that support scalability, reusability, and business-driven data ownership.
- Collaborate with business stakeholders, data engineers, architects, platform teams, and reporting consumers to ensure alignment throughout the transformation journey.
- Provide architectural leadership and recommendations on cloud infrastructure, technology stack, performance optimization, and modernization best practices.

**Required Skills & Experience**

- Experience in Data Architecture, Data Warehousing, and Enterprise Data Platforms.
- Strong expertise in SQL Server Data Warehousing, including complex SQL development, stored procedures, ETL frameworks, and dimensional modeling.
- Proven experience planning and delivering large-scale cloud migration and data modernization initiatives.
- Strong hands-on knowledge of AWS data services and Databricks architecture.
- Experience creating enterprise data lineage, impact assessment, dependency analysis, and migration mapping.
- Practical experience leveraging Generative AI and AI-assisted engineering tools to automate analysis, design, and migration activities.
- Strong understanding of Data Products, Data Mesh, Data Governance, and Modern Data Architecture principles.
- Experience developing migration roadmaps, implementation waves, target operating models, and adoption strategies.
- Excellent stakeholder management, communication, and documentation skills.

**Preferred Qualifications**

- Experience with enterprise-scale banking, financial services, or highly regulated environments.
- Knowledge of modern data engineering frameworks and lakehouse architectures.
- AWS and/or Databricks certifications.
- Experience with metadata management, data cataloguing, and lineage platforms.