Pricing For Talent
Login Free Trial Book a Demo

Inference Software Engineer

New

Skills

Auto-scaling C++ Distributed Systems GPU Kernels Inference Systems Load Balancing Quantization Rust SGLang TensorRT-LLM

Join SpaceX as an Application Software Engineer focusing on inference systems. This role involves developing highly reliable and high-throughput inference systems while architecting scalable distributed infrastructure for model serving.

Key Responsibilities
  • Design and implement scalable distributed systems for model serving, including load balancing and auto-scaling.
  • Optimize latency for production workloads using GPU kernels and quantization techniques.
  • Develop high-concurrency serving solutions ensuring low tail latency and high uptime.
  • Manage end-to-end components such as routing, SDKs, rate limiting, and scaling.
  • Benchmark and accelerate various inference engines like SGLang, vLLM, and TensorRT-LLM.
Required Skills & Qualifications
  • Bachelor's degree in Computer Science, Engineering, or Mathematics, or 2+ years of software experience.
  • Experience in designing and maintaining scalable distributed systems.
  • 1+ year of full-stack or backend development with production systems.
  • Proficiency in Rust or C++ programming languages.
  • Familiarity with LLM inference engines and serving frameworks.
  • Deep understanding of low-level systems programming, including GPU kernels and quantization.

No forms. Your profile is generated instantly.

Job Type: Remote

Salary: Not Disclosed

Experience: Entry

Duration: Months

Share this job:

Similar Jobs

LLVM Compiler Developer

Posted 122 days ago

Develop and enhance LLVM and Clang based toolchain components

Collaborate with LLVM community for continuous integration

Back-end Bash C C++

LLVM Compiler Developer Role

Posted 122 days ago

Enhance and maintain LLVM/Clang-based toolchains

Support and optimize code for diverse platforms

Back-end Bash C++ Communication

LLVM Compiler Engineer Role

Posted 122 days ago

Enhance and implement LLVM toolchain components

Extend open source compilers for new platforms

Back-end Bash C++ Communication

Senior C++ Virtualization Developer

Posted 122 days ago

Integrate backup solutions with virtualization and cloud platforms

Ensure reliable cross-platform VM backup and restore operations

C++ Cloud Devops Linux

ML Engineer - AdTech

Posted 122 days ago

Design and implement ML systems|Apply optimization strategies|Collaborate with teams|Analyze data

r user behavior|Develop data

C++ Data Analysis Java Machine Learning

Senior Software Engineer - BizTech

Posted 122 days ago

Solve challenging problems for Airbnb and users

Remove friction from user journey

Android C++ Engineer Java

Staff Software Engineer

Posted 122 days ago

Revolutionize enterprise data operations through AI solutions.

Automate and accelerate data tasks for overworked data teams.

Ai Airflow Ansible Api Development

Senior Rust Engineer UK

Posted 122 days ago

Hire a remote senior engineer

Develop high-quality Rust software

Backend Development Code Review Communication Skills Distributed systems

JAX Research Engineer Role

Posted 122 days ago

Optimize and scale JAX frameworks for distributed training.

Enhance production inference systems for speech-to-text models.

C++ Python Rust

Distributed ML Optimization Engineer

Posted 122 days ago

Optimize distributed ML performance

Accelerate deep learning inference

C++ CUDA Python Pytorch

Remote Engineering Manager Role

Posted 122 days ago

Hire a remote engineering manager

Lead and mentor engineering teams

Agile Development Agile Methodologies Cloud Cloud Computing

Senior Backend Developer - Node.js

Posted 122 days ago

Develop new products and features to enhance user experience

Ensure platform scalability and responsiveness

Architecture C++ JavaScript MongoDB

Blockchain Integration Specialist

Posted 122 days ago

Build, improve, and maintain blockchain infrastructure

Develop Node/JS workers for blockchain API interfaces

Apis Blockchain Blockchain technology Ethereum

Software Engineer - Staff A

Posted 122 days ago

Promote a flexible remote or hybrid work culture

Encourage inclusivity and diversity in hiring and career development

AWS C++ Cloud Computing Code reviews

iOS Browser Engineer Remote

Posted 122 days ago

Enhance privacy and security for iOS users

Develop and maintain Brave's iOS browser

C++ Engineer iOS JavaScript

iOS Browser Engineer

Posted 122 days ago

Develop and maintain Brave iOS browser

Ensure privacy and security for users

C++ Engineer Instrumentation iOS

Embedded Linux Field Engineer

Posted 122 days ago

Hiring a Mandarin-speaking Embedded Linux Field Engineer for mission-critical industries.

Engage customers during presales and propose solutions.

C C++ Debian Devops

Embedded Linux Senior Software Engineer

Posted 122 days ago

Optimise Ubuntu for IoT and server platforms

Work with partners to enhance Ubuntu experience

C C++ Docker IOT

Linux Engineer - Next-Gen Ubuntu

Posted 122 days ago

Optimizing Ubuntu for new silicon technologies

Maintaining high-quality software standards

C C++ Devops Engineer

C++/Rust Graphics Engineer

Posted 122 days ago

Design and implement features in the Mir stack

Participate in code reviews and testing

C++ Engineer Flutter Linux

Senior Juju Software Engineer (Go)

Posted 122 days ago

Seeking a Senior Software Engineer proficient in Go for Canonical's Juju software

Responsibilities include design, delivery, support, collaboration, coaching, and development

Ansible C++ Devops Docker

Software Support Engineer

Posted 122 days ago

Resolve complex customer problems related to Ubuntu, Kernel, Ceph, OpenStack, or Kubernetes

Maintain relationships with various teams

C C++ Ceph Devops

Software Engineer - L3 Support

Posted 122 days ago

Resolve complex customer problems related to open-source technologies

Maintain relationships with internal and external teams

C++ Ceph Debian packaging Devops

Software Engineer, Cloud - Sustaining Engineering

Posted 122 days ago

Resolve complex customer problems related to Ubuntu, OpenStack, Kubernetes, and open source software

Maintain close working relationships with field, support, and product engineering teams

C++ Devops Engineer Go

Graduate Software Engineer - Open Source

Posted 122 days ago

Hiring Graduate Software Engineers for open source projects

Engaging with the open source community and users

Code reviews Devops Engineer Go

IoT Solutions Architecture Manager

Posted 122 days ago

Lead and manage a team of Field engineers

Solve customer problems as a trusted advisor

Architecture C C++ Cloud

Solutions Architect for Automotive

Posted 122 days ago

Hiring a Solutions Architect with Automotive expertise

Engaging customers during presales and deliveries

C C++ Debian Devops

Open Source Network Software Engineer

Posted 122 days ago

Develop cutting-edge open source networking software

Optimize Ubuntu and related applications for ASICs and boards

Devops eBPF Engineer Go

Ubuntu Engineer

Posted 122 days ago

Design, build, and ship software in various programming languages.

Integrate, test, and manage open source software in Ubuntu repositories.

Architecture Build Systems Devops Engineer

Ubuntu Engineering Lead

Posted 122 days ago

Lead and develop high performing engineering teams

Define vision and strategy for Ubuntu

Architecture Communication Skills Devops Flutter

Junior Ubuntu Engineer

Posted 122 days ago

Hiring junior engineers for Ubuntu Engineering organization

Engage with and contribute to the open source community

Debian packaging Devops Engineer Flutter

Ubuntu Eng. Manager

Posted 122 days ago

Hiring engineering leaders for Ubuntu Engineering organization.

Managing and developing high-performing engineering teams.

Architecture Devops Flutter Go

MLOps Field Engineer

Posted 122 days ago

Designing ML architectures for external customers

Delivering AI/ML solutions in collaboration with sales leads

Big Data Devops Docker Engineer

Embedded Linux Engineer IoT

Posted 122 days ago

Expand reach in mission-critical industries through IoT solutions

Engage customers during presales for technology explanation

C C++ Debian Devops

Cloud Eng Manager @ Canonical

Posted 122 days ago

Manage a growing engineering team

Implement disciplined engineering and operations processes

Agile Agile Development C C++

OpenStack Eng. Manager

Posted 122 days ago

Lead and manage engineering team for optimal performance

Ensure team health and effective operations

Agile C C++ Cloud

Golang Engineer at Canonical

Posted 122 days ago

Hiring Golang engineers for all relevant teams at Canonical

Building systems delivering Ubuntu to the world using Golang

Apis Architecture C C++

Security Software Engineer

Posted 122 days ago

Challenging product teams to think deeply about security

Contributing to various aspects of product security

C C++ Cryptography devsecops

Lead Linux Kernel Engineer - Ubuntu

Posted 122 days ago

Lead engagements with partners for optimal solutions

Enhance performance and security of Ubuntu Linux kernel

Architecture Bash Scripting Device Drivers Devops

Python Engineer

Posted 122 days ago

Hiring Python Engineers at all levels

Working on open-source software and operating systems

C++ Docker Engineer Go

Rust Engineering Lead

Posted 122 days ago

Drive Rust adoption in Canonical and upstream projects

Design and implement software in Rust for Linux systems

C C++ Cloud Computer science

Cloud Engineering Manager - BootStack Team

Posted 122 days ago

Hiring a Cloud Engineering Manager for the BootStack team

Managing day-to-day operations and customer service

Agile software development C++ Cloud Cloud Technologies

Ubuntu Security Engineer

Posted 122 days ago

Protecting the open-source community and Ubuntu users from emerging threats

Collaborating with internal teams and external partners to address vulnerabilities

C Cloud Devops Engineer

Linux Platform Integration Engineer

Posted 122 days ago

Design and implement Ubuntu integration for IoT and server-class hardware

Optimize Ubuntu experience on partner platforms

C C++ Ci Automation Devops

Linux Systems Engineer - Optimisation

Posted 122 days ago

Optimizing Ubuntu for latest hardware platforms

Working on complex customer engagements

C C++ C programming Debian

Containerization & Virtualisation Engineer

Posted 122 days ago

Design and develop next-generation open source VMM or container technology

Collaborate with distributed teams on software delivery and security

Cloud infrastructure Devops Docker Engineer

Go Software Engineer

Posted 122 days ago

Design user experience for new tool capabilities

Write high-quality code to create new features

Ai/ml C C++ Devops

Engineering Manager - Ubuntu

Posted 122 days ago

Lead and mentor a team of engineers

Optimize Ubuntu for latest hardware platforms

Architecture C C++ Devops

Senior Engineer at Canonical

Posted 122 days ago

Leading design and architecture of impactful open source projects

Collaborating with a distributed team

Docker Engineer Go Performance Engineering

Performance Engineer - Open Source

Posted 122 days ago

Drive performance engineering skills and culture at Canonical

Provide insights and analysis on software performance and efficiency

Devops Engineer Go Java