Чтобы адаптировать резюме под вакансию или составить сопроводительное письмо, загрузи резюме
описание
Cast AI does not provide any form of visa sponsorship/work permit.
Cast AI is an automation platform that operates cloud-native and AI infrastructure at scale. It uses autonomous decision-making in Kubernetes and cloud environments to continuously optimize production performance, reliability, and efficiency. Its Kimchi system automatically matches workloads to cost-efficient, high-performing LLM and serving configurations on customer infrastructure.
задачи
Increase throughput through continuous batching, speculative decoding, chunked prefill, and kernel-level tuning across vLLM, SGLang, and TensorRT-LLM
Reduce TTFT and TPOT latency by profiling bottlenecks and addressing compute, memory bandwidth, scheduling, or networking issues
Improve KV cache utilization through paged attention, prefix caching, eviction policies, cache reuse, and quantized KV
Quantize weights, activations, and KV using INT8, INT4, and FP8 while measuring quality on real workloads
Reduce cold starts and memory footprint through faster initialization, smarter weight loading, and tighter memory accounting
Scale inference across nodes using distributed topologies, network-aware placement, and checkpointing strategies
Set the technical direction by deciding what to benchmark, adopt, and build, and align the team through writeups and reproducible experiments
требования
5+ Years building production ML systems
A portfolio demonstrating depth in inference or training infrastructure, not just model training notebooks
Strong Python skills for production services
Hands-on experience with at least one of vLLM, SGLang, or TensorRT-LLM and an understanding of inference engine performance on different GPUs
Fluency with quantization tradeoffs, including measuring quality regressions
Experience with distributed systems, collective communication, sharding strategies, and failure modes in multi-GPU and multi-node setups
A measurement-focused approach: instrument before optimizing and distinguish real improvements from benchmark artifacts
Self-direction and comfort with a broad mandate
условия
Competitive salary depending on experience level
Equity options
10% Of work time for personal projects or self-improvement
Learning budget for professional and personal development, including international conferences and courses
Annual hackathon
Team-building budget and company events
Equipment budget
Extra days off
No visa sponsorship or work permit is provided
A background check may be conducted at the final recruitment stage through Checkr
