Member of Technical Staff - ML Systems & Inference

About the role

As a Member of Technical Staff focused on ML Systems, you will build the inference systems that execute models end-to-end in production.

You will work on the systems that determine how inference executes across that pipeline: how requests are batched and scheduled, how stages are placed and scaled, how KV cache and intermediate state move between accelerators, and how the system balances latency, throughput, and utilization across different hardware characteristics.

You will work across model serving, batching, scheduling, concurrency, KV cache management, and memory placement. You will help bring up models on novel hardware. You will support new model architectures and inference techniques, improve performance under real production workloads, and partner with compiler, kernel, networking, and distributed systems engineers to optimize the full execution path.

What success looks like

In the first 12-18 months, you will:

• Improve the latency, throughput, and efficiency of production inference workloads
• Design execution strategies across batching, scheduling, concurrency, and resource utilization
• Improve KV cache management, memory efficiency, and execution under load
• Enable new models, accelerator architectures, and inference techniques to run efficiently in production

You may be a good fit if you have

• Strong software engineering fundamentals
• Experience building or operating ML inference or model serving systems
• Comfort reasoning about performance, memory usage, and system behavior under load
• Bachelor's degree in a relevant field, or an equivalent combination of education, training, and professional experience.

Strong candidates may also have

• Experience with inference runtimes such as TensorRT-LLM, vLLM, or custom serving systems
• Deep understanding of modern model architectures and attention mechanisms
• Experience with batching, scheduling, and concurrency control in inference systems
• Familiarity with KV cache management and memory placement strategies
• Experience profiling and tuning latency- and throughput-critical systems
• Software development experience in Python and C++