Data Engineer

Core is a high-growth Microsoft Managed Services Provider (MSP) delivering modern technology solutions across Azure, Microsoft 365, Security, and Modern Workplace. We specialise in providing a wide range of services, solutions and software to optimise our clients’ Microsoft investments. Our customers trust us to help them unlock productivity, reduce risk, and transform their businesses through technology.

We live by our CORE values: Customers first (we put customers at the heart of what we do and take accountability for our actions), One Core (we recruit, develop and retain great talent in a supportive team environment), Responsive (we’re flexible, approachable and responsive to customer needs), and Expertise (we’re proudly professional and continuously strive for excellence for customers, peers and partners).

Job description:
We are seeking a highly skilled and motivated Data Engineer to lead the design, development, and delivery of modern data platforms and analytics solutions. This role combines hands-on technical expertise with strategic thinking, enabling clients to unlock insights and value from their data using Microsoft Fabric, Databricks, and the wider Azure data ecosystem.

You will work closely with stakeholders to understand business requirements, architect scalable data solutions, and guide delivery teams through implementation. The ideal candidate has deep experience building lakehouse platforms - with strong grounding in medallion architecture, incremental data processing, and resource management (Databricks clusters, Fabric capacity) - and a working knowledge of Power BI to support the reporting layer on top of the data they build.

The main working language is English. The candidate must be able to communicate, read technical documentation and perform day-to-day duties in English.

Key Responsibilities:

• Lead the design and architecture of end-to-end data platforms using Microsoft Fabric and/or Databricks.
• Translate business requirements into scalable data models and lakehouse architectures, following medallion (bronze/silver/gold) design principles.
• Design and implement incremental data processing patterns to keep pipelines efficient and cost-effective at scale.
• Build and optimize data pipelines using Fabric components (Data Factory, Lakehouse, Dataflows Gen2, Notebooks) and/or Databricks (Workflows, Delta Live Tables, Notebooks).
• Manage compute resources responsibly - sizing and scaling Databricks clusters, and managing Fabric capacity - to balance performance and cost.
• Collaborate with BI developers and business analysts to ensure curated data supports accurate, trustworthy reporting, including in Power BI.
• Support presales activities including solution design, effort estimation, and client presentations.
• Establish and promote best practices in data governance, performance tuning, and DevOps/CI-CD for data platforms.
  

Required Skills & Experience
Essential:

• Proven experience designing and delivering enterprise-grade data pipelines and lakehouse architectures.
• Strong understanding of medallion architecture and incremental/delta processing patterns.
• Hands-on experience with Microsoft Fabric (Data Factory, Lakehouse, Dataflows Gen2, Notebooks) - including Fabric capacity management.
• Hands-on experience with Databricks (notebooks, workflows, cluster configuration and resource management).
• Experience with Azure SQL, Azure Storage Accounts, Azure Key Vault, and building orchestration pipelines.
• Working knowledge of Power BI (semantic models, DAX, Power Query) as a distinct skill supporting the reporting layer.
• Solid grasp of data pipeline best practices: modular and metadata-driven design, version control, and CI/CD.

Desirable:

• Exposure to Fabric IQ (Microsoft Fabric) and Genie (Databricks) for enabling AI-driven data analysis.
• Experience with other low-code or analytics platforms.
• Knowledge of data governance frameworks and security models.
• Relevant certifications (e.g., Fabric Data Engineer Associate / DP-700, Azure Databricks Data Engineer Associate / DP-750).

We offer:

• Flexible remote working arrangements.

• Work from home contract but alternatively, you can also work from our offices (London, Manchester, Edinburgh or Gdańsk (Poland)).
• Appropriate, modern technology to support the fulfilment of your role.
• Significant focus on Learning + Development, including training courses, seminars, and social events in London.
• Supportive, ambitious and entrepreneurial culture where you'll have the freedom to drive ideas, take ownership and make a real impact.
• Two paid leave days per year to volunteer for causes that resonate with Core's values, such as charitable efforts, environmental initiatives, and supporting the next generation in tech.
• Private healthcare.
• Employee Assistance Programme (EAP) for mental health and wellbeing. (UK)
• Monthly WFH allowance.
• Life Assurance: 5x annual salary.
• Cycle-to-work scheme. (UK)
• Multisport card. (PL)
• Catered quarterly & annual social events.
• Travel and accommodation allowance.