Senior Data Engineer

New

Skills

Apache Airflow Apache Spark Cost Optimization Databricks Data Orchestration Delta Lake Parquet S3 Trino Unity Catalog

We are seeking a Senior Software Engineer to design and build scalable Spark and Airflow platform primitives. You will lead the migration of Spark workloads to serverless Databricks and Delta Lake while optimizing compute resources to reduce costs at scale.

Key Responsibilities
  • Design and build scalable Spark and Airflow platform primitives.
  • Lead migration of Spark workloads to serverless Databricks and Delta Lake.
  • Optimize compute resources to reduce costs at scale.
  • Collaborate with analytics and product teams for self-serve data.
  • Improve reliability and governance with Unity Catalog and Trino.
  • Own multi-team platform reliability and cost optimization.
  • Engage in challenging, high-impact work to grow your career.
Required Skills & Qualifications
  • Extensive experience with large-scale Spark and Databricks platform infrastructure.
  • Deep expertise in data orchestration with Airflow.
  • Strong lakehouse fundamentals: S3 data lakes, Delta Lake, Parquet.
  • Familiarity with Trino, Pinot, or Hive Metastore.
  • Performance-based compensation with bonuses, equity, and 401(k) matching.
  • Top-tier benefits including 100% employee health coverage and family option.

No forms. Your profile is generated instantly.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: Months

Share this job:

Similar Jobs

Forward Deployed Software Engineer

Posted 60 days ago

Lead technical implementation and optimization of data platform

Serve as primary technical contact for key accounts

Airflow AWS Databricks Engineer

Staff ML Engineer, Apollo

Posted 60 days ago

Lead development of scalable ML systems

Advance Apollo's AI-native product features

Airflow Architecture Databricks Engineer

Senior ML Engineer, Remote

Posted 60 days ago

Design and productionize scalable machine learning systems

Personalize user experiences using data-driven models

Cloud Computer science Databricks Engineer

Senior ML Engineer II at Apollo

Posted 60 days ago

Build and productionize Machine Learning models for Apollo products

Optimize users' experience at all stages of their product journey

Airflow Ai Systems Cloud Computer science

ML Engineer on Apollo Team

Posted 60 days ago

Build and deploy ML models for Apollo products.

Enhance user experience through data-driven insights.

Airflow Cloud Computer science Databricks

Remote Sales Engineer

Posted 60 days ago

- Drive sales of modern data orchestration solutions - Build and maintain customer relationships -

rovide technical expertise to customers - Work collaboratively with the sales team - Achieve sales

Communication Crm Customer Relations Data Orchestration

Senior Analytics Engineer

Posted 60 days ago

Hiring a Senior Analytics Engineer remotely

Axios - Smart brevity

Airflow Analytics BigQuery Bi tools

Principal Data Engineer

Posted 60 days ago

Set technical direction for data initiatives

Design and build scalable data pipelines

Architecture Databricks Data Compliance Data Modeling

Principal Product Manager - AI Platform

Posted 60 days ago

Lead and drive product initiatives for AI and Data Product Platform

Adopt and implement data mesh framework across the organization

Agile Agile Development Databricks Financial Acumen

Principal Product Manager

Posted 60 days ago

Lead multiple product initiatives from concept to delivery.

Drive data business transformation and enhance customer outcomes.

Agile Agile Development Analytics Databricks

Senior Data Architect for Blackbaud

Posted 60 days ago

Lead data strategy and architecture for Blackbaud

Design breakthrough products in Data Intelligence

Big Data Databricks Data Modeling Machine Learning

Enterprise Data Architect Role

Posted 60 days ago

Design and oversee enterprise-wide data architecture

Develop strategies for data platforms and analytics

AWS Big Data Databricks Data Modeling

Principal Data Engineer Role

Posted 60 days ago

Modernize and optimize legacy data warehouse systems

Design and implement scalable data pipelines

Architecture AWS Databricks Devops

Senior AI/ML Engineer

Posted 60 days ago

Design and optimize scalable AI/ML models

Integrate AI solutions across business units

Databricks Hadoop Java Kafka

Principal Product Manager, Data

Posted 60 days ago

Lead strategic product initiatives for data and AI platforms

Drive organizational adoption of data mesh framework

Agile Agile Methodologies Databricks Financial Acumen

AI/ML Senior Engineer Role

Posted 60 days ago

Design and optimize scalable AI/ML models

Integrate AI solutions across business units

Databricks Engineer Hadoop Java

Senior Data Engineer Role

Posted 60 days ago

Design and maintain scalable data pipelines

Ensure data quality, reliability, and performance

Airflow Apache Kafka AWS Big Data

Senior Data Engineering Manager

Posted 60 days ago

Lead and manage distributed data engineering teams

Define and execute strategic direction for data platforms

BigQuery Databricks Data Engineering Devops

Sr Data Engineer

Posted 60 days ago

Design and develop best-in-class cloud platform

Develop solutions across data and analytics stack

BigQuery Databricks Devops Engineer

Staff Data Engineer

Posted 60 days ago

Lead design and development of data architecture

Implement scalable and reliable data pipelines

AWS Big Data Databricks Devops

Sr. Analytics Engineer at Hims & Hers

Posted 60 days ago

Build scalable data models and manage data transformations

Collaborate with business partners to understand and solve data needs

Databricks Redshift Snowflake SQL

Senior Solutions Architect

Posted 60 days ago

Drive growth for Chainlink through decentralized oracle networks.

Provide technical guidance to sales, product, and integrations teams.

AWS Blockchain technology DApps Databricks

Senior Data Engineer India

Posted 60 days ago

Deliver scalable data engineering solutions for enterprise clients.

Design and build cloud-native data pipelines and server-side applications.

AWS BigQuery Databricks Engineer

Lead Data Engineer Role

Posted 60 days ago

Architect scalable, secure data platforms.

Implement modern software engineering practices.

Airflow AWS Aws glue Databricks

Senior Data Lead Opportunity

Posted 60 days ago

Lead and mentor data engineering teams

Architect and deliver scalable cloud data solutions

Apache Kafka AWS Big Data Databricks

Data Solutions Architect

Posted 60 days ago

Lead and mentor engineers in designing technical solutions

Ensure performance, security, scalability, and data integration

AWS Databricks Google Cloud Platform Java

Lead Solutions Architect

Posted 60 days ago

Lead and mentor a team of engineers

Design and implement end-to-end technical solutions

Databricks Google Cloud Platform Hadoop Java

Principal Solution Architect

Posted 60 days ago

Propose, design, and provision cloud-native data solutions.

Lead a technical team operating modern data platforms.

Airflow AWS Cloud Computer science

Managed Services Architect

Posted 60 days ago

Deliver enterprise data platform solutions

Administer and optimize big data environments

AWS Big Data Databricks Hadoop

Senior Solutions Architect Role

Posted 60 days ago

Design and implement scalable data architectures

Lead and mentor engineering teams

Airflow AWS Databricks Hadoop

Solutions Architect Managed Services

Posted 60 days ago

Design and implement scalable data platforms

Lead and deliver cloud and big data projects

AWS Big Data Databricks Devops

Lead Data Engineering Role

Posted 60 days ago

Lead and mentor data engineering teams

Design and implement scalable data solutions

Apache Kafka AWS Databricks Devops

Senior Data Engineer Remote

Posted 60 days ago

Operate and support cloud data platforms

Resolve and prevent data pipeline issues

AWS Cloud Databricks Engineer

Data Platform Program Manager

Posted 60 days ago

Lead enterprise-wide data platform migration programs

Ensure compliance with healthcare data regulations

Agile Project Management Architecture AWS Budget Management

Azure Databricks Data Architect

Posted 60 days ago

Design and implement scalable Databricks data architectures

Lead migration from legacy data systems to Lakehouse platforms

Architecture Databricks Docker Microsoft Azure

QA Lead Data Engineering

Posted 60 days ago

Establish robust QA strategies for data platforms

Lead testing of data migrations to Databricks

Agile Agile testing Automation Databricks

Database Project Manager

Posted 60 days ago

Lead and manage client engagements on database and data engineering projects.

Translate client needs into technical requirements.

Cloud Cloud Platforms Communication Databricks

Senior QA Lead - Data Platforms

Posted 60 days ago

Lead QA strategy and execution for data platforms

Establish testing frameworks and methodologies

Agile Agile testing Automation Compliance standards

AWS Migration Engineer

Posted 60 days ago

Lead and execute AWS migration projects

Design and implement migration strategies

AWS Cloud Architecture Devops Docker

Data Quality Software Engineering

Posted 60 days ago

Drive fast delivery of product features through effective engineering leadership.

Ensure alignment between architectural decisions and the product roadmap.

Api Integration Data Data Orchestration Devops

Software Engineer (Data Engineering)

Posted 60 days ago

Building and scaling data pipelines for large volumes of data processing

Implementing ETL processes for data integration

Databricks Data Engineering Dataflow EMR

Data Engineer Role

Posted 60 days ago

- Design and implement scalable data pipelines with AWS and Databricks - Optimize relational and

-relational databases using SQL and MongoDB - Utilize Kafka for real-time data streaming and

AWS Databricks Docker Kafka

AI Data Engineer Project

Posted 60 days ago

At least 4-5 years of experience in Data Engineering, Expertise in Azure cloud service platform.

Good knowledge in Azure services : Azure Data Factory, Azure SQL Database, Azure Key Vault, Azure OpenAI Service.

Automotive Industry Databricks Data Modeling English Proficiency

Code.org Student Learning Project

Posted 60 days ago

Develop AI tools and React apps for student learning

Collaborate with product managers, designers, and curriculum writers

Ai technology AWS JavaScript LLMs

Mid-Level Full-Stack Developer

Posted 60 days ago

Contribute to Operations and Maintenance of mission critical work

Communicate with stakeholders and project managers

Elastic Search JavaScript Node.js S3