Data Engineer - Data Lake

New

Skills

Airflow Apache Spark AWS (EMR/Glue/S3/Athena/Redshift) Cloud-native systems Data Lake architecture Data quality management Docker ETL/ELT pipelines Kubernetes Python (PySpark)

We are seeking a skilled Staff Data Engineer to architect, build, and scale distributed ETL/ELT pipelines across diverse healthcare datasets. This role involves leading Data Lake architecture focused on scalability, observability, reliability, and cost control, while ensuring high data quality and standardization.

Key Responsibilities
  • Design and optimize batch and near real-time data processing with cloud-native systems.
  • Mentor engineers and drive production reliability across the platform.
  • Collaborate with Product, Infrastructure, Security, and downstream teams to ensure secure data delivery.
Required Skills & Qualifications
  • 8+ years in data engineering or related fields.
  • Strong proficiency in Python (PySpark), Java, or Scala, with advanced SQL skills.
  • Deep experience with Apache Spark and AWS big data services.
  • Experience with Airflow or Argo, and distributed storage formats like Parquet.

No forms. Your profile is generated instantly.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: Months

Share this job:

Similar Jobs

Senior Full Stack Engineer

Posted 81 days ago

Develop and maintain full stack web and mobile applications.

Implement and automate robust backend API services.

Angular API Design AWS CI/CD

Software Engineer at Acquia

Posted 81 days ago

Develop and maintain customer-facing applications and microservices on Kubernetes

Collaborate with cross-functional teams to document product and development details

Ansible AWS Devops Engineer

Senior Go Cloud Developer

Posted 81 days ago

Develop scalable cloud disaster recovery services

Design and implement high-load distributed systems

Algorithms Architecture Cloud Cloud Services

Principal AI Engineer Role

Posted 81 days ago

Hire a remote Principal AI Engineer

Develop customer experience automation solutions

Ai Automation AWS Cloud Computing

Affirm Senior Backend Engineer

Posted 81 days ago

Lead a team of engineers through ambiguity to solve problems

Collaborate with various stakeholders in product development

Android AWS Code Review Distributed systems

Software Engineer II, Backend

Posted 81 days ago

Develop backend systems for credit decisioning.

Collaborate with team members and stakeholders.

Android AWS Backend Systems Code Review

Forward Deployed Software Engineer

Posted 81 days ago

Lead technical implementation and optimization of data platform

Serve as primary technical contact for key accounts

Airflow AWS Databricks Engineer

Staff AI Backend Engineer

Posted 81 days ago

Architect and maintain high-performance backend infrastructure

Integrate advanced AI and LLM technologies into data workflows

AWS Docker Engineer FastAPI

Staff Software Engineer Role

Posted 81 days ago

Lead backend development for AI-powered data solutions

Architect scalable systems and APIs for enterprise clients

Agile Methodologies Angular AWS CI/CD

Staff Software Engineer, AI Backend

Posted 81 days ago

Design and build scalable AI-driven backend systems

Integrate advanced language models into data workflows

AWS Docker FastAPI Google Cloud Platform

Staff Software Engineer

Posted 81 days ago

Revolutionize enterprise data operations through AI solutions.

Automate and accelerate data tasks for overworked data teams.

Ai Airflow Ansible Api Development

AI-Powered Data Operations Revolution

Posted 81 days ago

Revolutionize enterprise data operations through AI automation.

Develop high-performance backend systems for AI solutions.

Docker Engineer FastAPI Python

Data Lead: ETL & Analytics

Posted 81 days ago

Improve data infrastructure Optimize performance and accessibility Enable data-driven

g Collaborate with cross-functional teams Mentor and lead data

Airflow Analytics AWS BigQuery

Anaplan Full Stack Engineer

Posted 81 days ago

Build exceptional software for a global growth strategy

Deliver seamless user experiences through architecture and collaboration

Android Css Docker Engineer

Angular Frontend Technical Lead

Posted 81 days ago

Lead technical delivery of Angular frontend

Ensure compliance with financial regulations

Angular Ci/cd Pipelines Css3 Docker

Angular Frontend Tech Lead

Posted 81 days ago

Lead frontend delivery for a banking risk management tool

Ensure technical excellence, scalability, and compliance

Angular CI/CD Docker Git

Senior Node.js Angular Engineer

Posted 81 days ago

Deliver high-quality software using Node.js and Angular.

Lead and mentor engineering teams in Agile environments.

Angular AWS CI/CD Docker

Senior Angular Frontend Engineer

Posted 81 days ago

Redesign and optimize financial applications

Deliver high-quality, performant frontend solutions

Angular CI/CD Css Docker

Senior Frontend Engineer (Angular)

Posted 81 days ago

Enhancing operations in the financial sector through IT solutions.

Developing client-specific software solutions.

Angular CI/CD Css Css3

Backend Engineer

Posted 81 days ago

Design and implement scalable backend endpoints

Mentor team members on best practices

Ansible Api Development AWS Docker

Senior Backend Engineer

Posted 81 days ago

Develop scalable backend solutions

Mentor team members

Angular Api Development Architecture AWS

Senior Backend Engineer Role

Posted 81 days ago

Design scalable backend solutions

Lead full software development lifecycle

Agile Methodologies Android Android development Apache Kafka

Senior Frontend Tech Lead

Posted 81 days ago

Architect and evolve scalable frontend platforms

Lead cross-functional initiatives to drive platform adoption

Css Cypress Docker Engineer

Engineering Manager, Apollo

Posted 81 days ago

Lead and mentor engineering teams

Drive technical excellence and process improvement

Architecture Docker Elasticsearch Kubernetes

Senior Frontend Engineer Role

Posted 81 days ago

Drive high-quality frontend development

Collaborate cross-functionally with teams

Agile Methodologies Angular Css Cypress

Staff ML Engineer, Apollo

Posted 81 days ago

Lead development of scalable ML systems

Advance Apollo's AI-native product features

Airflow Architecture Databricks Engineer

Engineering Manager Role

Posted 81 days ago

Lead and mentor engineering teams

Drive process and cultural excellence

Architecture Docker Elasticsearch Kubernetes

Backend Engineer Role Overview

Posted 81 days ago

Develop scalable backend solutions

Collaborate cross-functionally with teams

Docker Elasticsearch Engineer Kubernetes

Quality Engineer Automation Specialist

Posted 81 days ago

Drive automation testing initiatives across engineering teams

Ensure software quality through best practices and robust methodologies

Automation Computer science Cypress Engineer

Engineering Manager

Posted 81 days ago

Lead engineering team towards excellence and growth

Mentor team to accelerate career development

Cloud Cloud-native architecture Css Data Infrastructure

Senior ML Engineer II at Apollo

Posted 81 days ago

Build and productionize Machine Learning models for Apollo products

Optimize users' experience at all stages of their product journey

Airflow Ai Systems Cloud Computer science

Senior Frontend Engineer

Posted 81 days ago

Implement best practices

Drive quality and innovation

Android AWS CI/CD Cloud

Senior Fullstack Engineer

Posted 81 days ago

Work with Apollo’s 200M+ Leads data and build features on top of it

Improve data quality and build customer-facing products

Android API Design Automated Testing AWS

Sr. Fullstack Engineer at Apollo

Posted 81 days ago

Work on high-impact features using a diverse tech stack

Collaborate with cross-functional teams for product development

Elasticsearch Front end Google Cloud Platform JavaScript

Collaborative Frontend Engineer

Posted 81 days ago

Drive best practices in technology and processes

Collaborate with cross-functional teams

Docker Elasticsearch Engineer Front end

ML Engineer on Apollo Team

Posted 81 days ago

Build and deploy ML models for Apollo products.

Enhance user experience through data-driven insights.

Airflow Cloud Computer science Databricks

Senior Frontend Engineer Project

Posted 81 days ago

Drive best practices and innovation in technology and architecture

Design and implement core product initiatives

Css Cypress Docker Front end

Scalable Backend Engineer

Posted 81 days ago

Implement best technology practices

Drive software development lifecycle

Ansible Docker Engineer MongoDB

Apollo Fullstack Engineer

Posted 81 days ago

Improve data quality and build customer-facing products

Drive best practices in technology, architecture, and process

Css Elasticsearch Engineer Front end

Data Architect Role

Posted 81 days ago

Design scalable data architectures, Lead big data systems development, Integrate AI and

ion, Deploy production services, Drive thought

AWS Docker Gitlab Hadoop

Sr. Software Engineer - LLM Team

Posted 81 days ago

Provide technical solutions and product improvements based on customer needs

Build features and integrations within the LLM gateway service

Ci/cd Pipelines Cloud infrastructure Data structures and algorithms Docker

AI Platform Lead Engineer

Posted 81 days ago

Design scalable AI data platforms

Optimize ML pipeline efficiency and resource allocation

Airflow BigQuery Dataflow Pyspark

Remote Backend Engineer Poland

Posted 81 days ago

Hire a remote backend engineer

Enhance team collaboration tools

Android AWS Docker Engineer

Systems Engineer - WordPress VIP

Posted 81 days ago

Recruiting Systems Engineers for WordPress VIP

Building and maintaining global infrastructure

Devops Docker ELK Engineer

WordPress VIP Systems Engineer

Posted 81 days ago

Develop and maintain scalable infrastructure.

Enhance system performance and fault tolerance.

Devops Docker ELK Engineer

Senior Backend Developer Project

Posted 81 days ago

Develop scalable and high-performance applications using Node.js and NestJS.

Collaborate with multidisciplinary teams to design and implement robust solutions.

API Design Devops Docker Elasticsearch

Senior Analytics Engineer

Posted 81 days ago

Hiring a Senior Analytics Engineer remotely

Axios - Smart brevity

Airflow Analytics BigQuery Bi tools

Remote Azure Job Openings

Posted 81 days ago

Enable remote cloud solution engineering

Ensure secure and scalable Azure environments

Ci/cd Pipelines Cloud Security Docker Kubernetes