Data Engineer III – Databricks & Python

Be part of a dynamic team where your distinctive skills will contribute to a winning culture and team.
 

As a Data Engineer III at JPMorganChase within the External Regulatory Financial Control (ERFC) Technology team, you will play a crucial role in designing, developing, and maintaining scalable data pipeline solutions using Databricks, Python/PySpark on AWS. You will collaborate with cross-functional teams to deliver high-quality data pipelines that support our business objectives.

Job responsibilities

• Design, develop, and maintain robust data pipelines using Python and PySpark on Databricks platform on AWS
• Process and transform large-scale financial datasets, implementing big data processing techniques to produce aggregated financial data for analytics and reporting
• Optimize complex queries and data processing workflows to ensure efficient performance at scale
• Analyze aggregated data outputs to identify data quality issues, anomalies, and processing bottlenecks, implementing corrective solutions
• Participate in the full Software Development Life Cycle (SDLC), including requirements gathering, design, development, testing, deployment, and maintenance
• Implement data quality checks, monitoring, and alerting mechanisms to ensure data accuracy and pipeline reliability
• Work with our partners Product Owners and end users to support their business use cases
• Act as both Production Support and SRE function as part of the Data Engineer role
• Utilise AI tools to quickly build and test new data pipelines (e.g. CoPilot, Claude Code)

Required qualifications, capabilities, and skills

• Strong hands-on experience in data engineering or related roles
• Strong proficiency in Python and PySpark for large-scale data processing
• Demonstrated experience with Databricks platform and Apache Spark ecosystem
• Proven track record of building and optimizing data pipelines for big data workloads
• Strong SQL skills with experience in query optimization and performance tuning
• Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.)
• Strong analytical skills with ability to investigate data issues, identify root causes, and implement solutions
• Experience with the complete SDLC, Jules/Jenkins, Spinnaker, Sonar and Agile methodologies
• Bachelor's degree in Computer Science, Engineering, Mathematics, or related technical field

Preferred qualifications, capabilities, and skills

• Experience working with financial data and understanding of data aggregation techniques
• Experience with data orchestration tools (Airflow, Step Functions, etc.)
• Understanding of financial services industry and regulatory requirements
• Databricks or AWS certifications
• Automated testing frameworks, e.g. Playwright, Cucumber, Gherkin etc.
• Experience with Parquet, JSON, CSV, Avro, Delta Lake