GPU Inference Systems Engineer

Ensigncode is hiring a GPU Inference Systems Engineer to own runtime and workload design: batching, grouped workloads, scheduling, and throughput/latency trade-offs across GPU inference pipelines. You'll integrate optimized kernels into real model pipelines and investigate end-to-end performance, not just isolated kernels.

A fast kernel in isolation doesn’t guarantee a fast system. As GPU Inference Systems Engineer, you’ll own the runtime and workload layer that connects our kernel work to real inference pipelines: batching strategy, scheduling, stream and concurrency management, and the operator-to-operator overhead that determines whether a pipeline’s end-to-end performance matches its individual kernels’ potential.

You’ll work closely with the kernel engineer and profiling engineer to turn optimized building blocks into a system that actually performs under real, variable production traffic, not just in a benchmark.

Responsibilities

• Design GPU inference execution flows, including batching, grouped workloads, scheduling, and throughput/latency trade-offs
• Integrate optimized kernels into model or inference pipelines; reduce overhead between operators and across execution stages
• Tune memory reuse, concurrency, streams, synchronization, and workload dispatch
• Investigate end-to-end performance rather than optimizing isolated kernels only

Core requirements

• C++ and CUDA; GPU inference concepts; batching and scheduling; experience with frameworks such as PyTorch or TensorRT is useful
• Ability to reason about latency, throughput, utilization, and workload variability

Team contribution

Connects low-level kernels to real workloads and defines the execution behavior the integration engineer exposes.

Works most closely with