Job Description:

Lead Data Engineer
Outside IR35
£500–£525 per day
3 Month Initial Contract
 
We're looking for an experienced Lead Data Engineer to join a high-profile project. This is a hybrid role, requiring 2 days per week onsite in London, with the remainder worked remotely.

What we're looking for:

• Proven experience leading data engineering teams, with a strong track record of mentoring and developing junior engineers.
• Expert knowledge of Python and Airbyte CDK, with experience building custom data connectors (a key focus of the role).
• Strong experience developing ELT pipelines using PySpark in Databricks within a Medallion Architecture.
• Experience with Terraform and deploying Airbyte connections and Databricks assets using GitHub Actions.
• Confident working with JIRA, GitHub, and VS Code.

 
Desirable:

• Experience with Databricks Delta Live Tables (DLT).
• Knowledge of Databricks Asset Bundles.