Senior Azure Databricks Engineer

601737
  • £80000 - £85000 per annum
  • Glasgow, Scotland
  • Permanent

Azure Data Architect / Senior Data Engineer (Hybrid - Glasgow, 3 Days Onsite)

Overview

A senior technical role within a multinational consultancy, working on behalf of a major client in the energy sector. The position focuses on designing, developing, and maintaining Azure-based data platforms-particularly Delta Lake and modern analytics services-while providing architectural leadership and hands‑on engineering expertise. The role operates on a hybrid model, requiring three days per week onsite in Glasgow.

Key Responsibilities

  • Develop and maintain Azure data architecture, including Delta Lake, ensuring compliance with data governance and security standards.

  • Analyse and resolve data integration and system interoperability challenges across complex environments.

  • Design and implement Azure data solutions using services such as Databricks, Azure SQL Database, Data Pipelines, Data Lake Store, Data Lake Analytics, Stream Analytics, and Azure Storage.

  • Lead the design and configuration of solutions within Azure Synapse and Databricks.

  • Provide architectural guidance and mentorship to Azure Data Engineers and project teams.

  • Review and enhance existing data models and pipelines to improve performance, stability, and interoperability.

  • Translate business requirements into robust data models through close collaboration with product management and SMEs.

  • Develop and maintain PySpark notebooks and SQL scripts for data transformation and analytics.

  • Build and optimise Synapse and Databricks pipelines for batch and streaming workloads.

  • Manage GitHub branching strategies and oversee CI/CD pipelines using GitHub Actions.

Personal Specification

  • Strong experience with Azure Synapse, Databricks, and Delta Lake architecture.

  • Proven ability to design scalable, cloud‑native data solutions.

  • Proficiency in PySpark, SQL, and distributed data processing.

  • Experience mentoring engineering teams and leading architectural decisions.

  • Strong analytical skills with the ability to assess and improve existing data designs.

  • Excellent communication skills and ability to work with business stakeholders.

  • Hands‑on experience with GitHub and CI/CD automation.

Hannah Farrow Team Lead

Apply for this role