Data Software Engineer, Azure Databricks

We are looking for a Data Software Engineer to help build a scalable solution used by data engineers to build, test, deploy and monitor data pipelines. You will develop reusable libraries and SDK modules that enable teams to ship data products autonomously and efficiently, working with Databricks, Delta Lake, Azure Data Lake and SAP HANA Data Lake.

Responsibilities

• Develop and maintain reusable SDK modules (Python) for data pipeline operations such as compaction, quality checking, schema evolution and table lifecycle management
• Design and build framework libraries following software engineering best practices, ensuring scalability and reliability across 100+ pipelines and diverse data domains
• Collaborate with Data Engineers who consume the framework to gather feedback, understand pain points and iterate on the SDK
• Contribute to platform architecture discussions and performance tuning strategies
• Write high-quality documentation and contribute to enablement materials for framework consumers
• Help define and infuse data engineering best practices through enablement, SDKs and templates
• Ensure data quality, consistency and governance across Lakehouse environments

Requirements

• Strong experience as a Data Engineer with proficiency in Databricks
• 3+ years of experience building reusable libraries, SDKs or internal developer tooling
• Knowledge of Data Mesh/Data Product concepts including data ownership, domain-oriented design and self-serve data platforms
• Deep expertise in Delta Lake, Delta tables and compaction with optimization for high-performance workloads
• Proven experience designing and maintaining complex data pipelines on cloud object stores (ADLS, etc.)
• Strong programming skills in Python for data engineering workloads
• Solid understanding of Lakehouse architecture and best practices for large-scale data platforms
• Hands-on experience with data pipeline monitoring, troubleshooting and performance tuning
• Familiarity with CI/CD and workflow orchestration (Databricks Jobs)
• Experience working in agile teams with a focus on ownership, autonomy and best practices
• Excellent problem-solving skills and capability to handle high-scale, complex data challenges
• English proficiency at B2 level or higher

Nice to have

• Experience with agile methodologies (Scrum, SAFe)
• Experience developing enablement materials and technical documentation

Benefits

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers
• Discounts in local language schools, including online courses for the Kazakh language

We cover it all:

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership