Senior Data Software Engineer/ AI Agents, Azure, Spark

We are looking for an experienced Senior Data Software Engineer with strong hands-on expertise in Azure and the Spark ecosystem, primarily focused on building and maintaining data transformation pipelines.

The candidate should be comfortable with any Spark-adjacent technology (PySpark, Spark SQL, Scala Spark, Databricks, Synapse, etc.) rather than being locked into one specific flavor. Experience with Microsoft Fabric is a strong plus but not a requirement.

The ideal candidate combines great technical skills with leadership capability, contributing to architecture, design, development, and mentoring of engineering teams — preferably in complex enterprise or financial services environments.

Responsibilities

• Lead the design, development, and optimization of scalable data engineering solutions on Azure, using Spark-based processing (PySpark, Spark SQL, Scala, or equivalent)
• Own end-to-end data transformation pipelines, including ingestion, transformation, storage, and analytics
• Work with Azure-native data services such as Data Factory, Databricks, and Synapse
• Support high-performance data access patterns using Cosmos DB (NoSQL API) where applicable
• Collaborate with data scientists, AI engineers, and product stakeholders to enable data-driven analytics and insights
• Mentor and guide junior engineers, setting coding standards and best practices
• Ensure data quality, security, governance, and performance across platforms
• Contribute to technical decision-making and solution architecture discussions

Requirements

• 3+ years of experience in data engineering roles, preferably within complex enterprise or financial services environments
• Expertise in Azure cloud data services, including Data Factory, Databricks, and Synapse
• Proficiency in the Spark ecosystem, including PySpark, Spark SQL, and Scala Spark
• Background in designing and maintaining end-to-end data transformation pipelines covering ingestion, transformation, storage, and analytics
• Skills in ensuring data quality, security, and governance across large-scale platforms
• Capability to contribute to solution architecture and technical decision-making
• Competency in mentoring engineering teams and setting coding standards and best practices
• Understanding of high-performance data access patterns using Cosmos DB (NoSQL API)
• English proficiency at an Upper-Intermediate level (B2) or higher

Nice to have

• Hands-on experience with Microsoft Fabric and OneLake (Delta / OpenLake)
• Familiarity with financial instruments and financial services data
• Exposure to AI-assisted development tools such as GitHub Copilot and awareness of industry-standard LLMs
• Knowledge of Data Science fundamentals and collaboration experience with DS teams

Benefits

We connect like-minded people

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

We cover it all

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Comprehensive medical & family care package
• Five trust days per year (sick leave without a medical certificate)
• Benefits package (sports activities, a variety of stores and services)

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers
• Discounts in local language schools, including online courses for the Kazakh language

We cover it all:

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

We cover it all:

• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Discounts for fitness clubs and sports programs
• Benefits package (sports activities, a variety of stores and services)

We connect like-minded people

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

We cover it all

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Comprehensive medical & family care package
• Four trust days per year for personal needs
• Discounts for fitness clubs
• Benefits package (hotels, restaurants, stores and services)

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers

• Discounts in local language schools, including offline courses for the Uzbek language

We cover it all:

• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Four trust days per year (sick leave without a medical certificate)
• Discounts for fitness clubs, dance schools and sports programs
• Benefits package (sports activities, a variety of stores and services)