Lead Data Software Engineer with Databricks

We are seeking a Lead Data Software Engineer with Databricks expertise to drive the technical design, robustness, and evolution of end-to-end traceability solutions, enabling transparency, regulatory compliance, and data-driven decision-making across the supply chain.

This role provides technical leadership and architectural direction for traceability platforms and data products, working with heterogeneous datasets ranging from farm and geolocation data to transactional, logistics, sustainability, and regulatory information.

The successful candidate will collaborate closely with the Traceability solution team, sustainability experts, business analysts, data engineers, and data scientists, acting as a technical reference to ensure traceability data pipelines, data models, and platforms remain aligned with best practices and long-term architectural vision.

Responsibilities

• Provide technical leadership and ownership for traceability data platforms, pipelines, and data products
• Define, document, and enforce architectural principles, technical standards, and best practices for traceability solutions
• Oversee the design of robust pipelines, quality controls, and proactive monitoring frameworks to ensure data availability, quality, integrity, and timeliness
• Guide and support data engineers and technical contributors on implementation choices, troubleshooting, performance optimization, and best practices
• Contribute strategically to Run execution and continuous improvement, including data quality, lineage, auditability, and operational monitoring
• Translate traceability and regulatory requirements into technical solutions, data models, and platform capabilities
• Collaborate closely with solution architects, data platform teams, and external partners to ensure alignment and scalability
• Support adoption of traceability solutions by business and sustainability teams through technical guidance and enablement

Requirements

• 5+ years of experience in technical roles within data, analytics, or platform environments, with exposure to leadership or solution ownership responsibilities
• Expertise in cloud-based data platforms, preferably Azure
• Background in data architecture, data modeling, and data platform design, including complex, large-scale datasets
• Hands-on experience with Databricks, Spark/PySpark, and SQL for data transformation pipelines and analytics platforms
• Proficiency in SQL and data storage concepts, including relational databases, NoSQL, and data lakes, with performance optimization for large and complex datasets
• Knowledge of Python and data manipulation ecosystems such as Pandas and NumPy, with ability to review and guide code even if not hands-on full time
• Familiarity with data visualization tools including Power BI, Tableau, and Looker
• Experience designing or governing data quality frameworks, lineage, monitoring, and auditability
• Familiarity with orchestration tools and data integration patterns such as ADF, Airflow, and AWS Glue, or equivalent
• Experience with software engineering best practices including version control with git, branching strategies, code reviews, and CI/CD
• Understanding of data governance, security, and access control for sensitive business and regulatory data
• Proficiency in English at a B2+ level

Benefits

We connect like-minded people

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

We cover it all

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Comprehensive medical & family care package
• Five trust days per year (sick leave without a medical certificate)
• Benefits package (sports activities, a variety of stores and services)

We connect like-minded people

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

We cover it all

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Comprehensive medical & family care package
• Four trust days per year for personal needs
• Discounts for fitness clubs
• Benefits package (hotels, restaurants, stores and services)

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers
• Discounts in local language schools, including online courses for the Kazakh language

We cover it all:

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

We cover it all:

• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Discounts for fitness clubs and sports programs
• Benefits package (sports activities, a variety of stores and services)

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

• Discounts in local language schools, including offline courses for the Uzbek language

We cover it all:

• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Four trust days per year (sick leave without a medical certificate)
• Discounts for fitness clubs, dance schools and sports programs
• Benefits package (sports activities, a variety of stores and services)