NDA

backend developer in LLM inference

грейд
senior
формат
удалённо
язык
Английский B1
опубликована
сегодня

описание

The client is building a real-time AI voice platform and developing large language model inference capabilities for production use.

задачи

  • Build and optimize a vLLM-based inference stack for low latency, high throughput, and maximum GPU utilization
  • Optimize continuous batching, scheduling, prefill/decode, prefix caching, and KV-cache allocation and reuse
  • Optimize or implement CUDA and Triton kernels for attention, GEMMs, sampling, normalization, and other critical model operations
  • Evaluate and integrate FlashAttention, FlashInfer, CUDA Graphs, torch.compile, speculative decoding, and quantization
  • Optimize distributed inference using tensor, data, and expert parallelism, NCCL, NVLink/NVSwitch, and InfiniBand
  • Work with researchers to bring new dense and MoE model architectures into production
  • Take ownership from profiling through production deployment

требования

  • Deep hands-on experience with LLM inference, ideally inside vLLM, SGLang, TensorRT-LLM, or similar systems, not just deploying them
  • Understanding of TTFT, inter-token latency, throughput, continuous batching, PagedAttention, KV caching, and prefill versus decode performance
  • Ability to profile GPUs and reason about compute, memory bandwidth, kernel launches, synchronization, and communication bottlenecks
  • Experience with CUDA, Triton, PyTorch, NCCL, and modern NVIDIA GPU architectures
  • Understanding of Transformer internals, including MHA/GQA, ROPE, KV cache, quantization, and MoE
  • Experience optimizing distributed, performance-critical systems in production
  • Care about performance, simplicity, and reliability

условия

  • Условий в вакансии нет

навыки

похожие вакансии

статьи для разработчиков