Lead Data Engineer

About Gradera

Gradera defines a new category of enterprise transformation called Software-Orchestrated Services™ - where software orchestrates human expertise, digital workers, and enterprise systems to deliver governed outcomes at scale. As an AI Native Services firm, we help enterprises redesign how work gets done across operations, product, engineering, customer experience, data, and enterprise workflows to move beyond fragmented AI pilots and disconnected automation toward measurable business outcomes.

Overview

We are seeking a hands-on, collaborative Lead Data Engineer to drive the technical direction, architecture, and delivery of our data foundation for digital twin and AI-powered enterprise platforms. As Lead Data Engineer, you will provide technical leadership and guidance to the data engineering team, ensuring the delivery of robust, scalable, high-performance data pipelines that power real-time operational intelligence and simulation-ready data products. This role partners closely with data architects, simulation engineers, ML engineers, and platform teams.

Our core data platform stack includes:

Data Platform & Lakehouse

• Databricks as the single point of truth for all data

• Databricks SQL for analytical queries

• Unity Catalog for metadata management and governance

• Terradata for data warehouse and business intelligence.

Stream & Event Processing

• Apache Kafka for real-time event ingestion and streaming

• Structured Streaming for continuous data processing

• Delta Live Tables for declarative, quality-enforced pipelines

Specialized Data Stores

• TimescaleDB for time-series operational data

Transformation & workflows

• Python, Scala, and SQL for data transformation and orchestration

• AI/ML workflows on Databricks models
  
  

Data Quality & Governance

• Great Expectations and Delta Live Tables expectations for data validation

• Unity Catalog for metadata management and compliance

• Work with Time‑series databases is a plus
  
  

Key Responsibilities

• Provide technical leadership, mentorship, and guidance to the data engineering team

• Lead the design, implementation, and evolution of data pipelines, Lakehouse architecture, and real-time event processing systems

• Architect and build data foundations that serve digital twin platforms, AI/ML workloads, and operational analytics

• Ensure delivery of high-quality, well-governed, and performant data products using Unity Catalog and Delta Lake

• Drive adoption of DataOps best practices, including CI/CD for data pipelines, automated testing, and monitoring

• Champion data hygiene, quality frameworks, and observability across the data estate

• Oversee real-time data ingestion from diverse operational systems and IoT sources

• Guide the use of modern data engineering tools, architectural patterns, and emerging technologies

• Own the implementation and quality of data lineage, cataloging, and governance to enable trust and compliance

• Collaborate with simulation engineers and ML teams to deliver simulation-ready and feature-engineered datasets

• Ensure engineering rigor, code quality, and documentation standards are met across the data stack

• Facilitate clear communication, knowledge sharing, and effective documentation within the team

• Support team growth through coaching, feedback, and skills development

• Drive development efficiency and alignment with product team priorities through clear prioritization frameworks and delivery metrics.
  

Preferred Qualifications

• 10+ years of hands-on data engineering experience, with 4+ years in a technical leadership role

• Track record of architecting and delivering scalable, production-grade data platforms

• Experience with modern DataOps practices (CI/CD for data, automated testing, pipeline monitoring)

• Demonstrated ability to mentor and grow data engineers

• Experience working in cross-functional, agile teams
  

Highly Desirable

• Experience building data foundations for digital twin or simulation platforms

• Familiarity with OpenUSD data pipelines or FMI/FMU data integration

• Experience with physics-informed or operational ML feature engineering

• Track record of delivering data platforms with sub-second latency for real-time operational use cases

• Experience thriving in fast-paced, ambiguous environments and balancing rapid delivery with technical excellence

• Experience leading or working with distributed, multidisciplinary teams

• Exposure to industrial domains such as Manufacturing, Logistics, or Transportation is a plus.