machine learning engineer for inference

Inworld is a research lab and inference provider focused on real-time AI for consumer-facing applications. It builds speech models, serves LLMs, and provides inference through modular APIs for high-volume, real-time workloads.

• Take models from the research team, containerize them, optimize their serving, and ensure they run reliably in production
• Ship stable engineering solutions and prioritize impact over theoretical optimizations
• Question and improve architectures to address latency and throughput challenges
• Design benchmarks or prototypes to investigate unclear problems
• Treat performance, latency, and reliability as first-class product features
• Share work and contribute to open source

• Strong understanding of modern serving frameworks and techniques such as vLLM or TRT-LLM
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
• Proficiency in C++, CUDA, Rust, or highly optimized Python
• Ability to profile code and optimize performance on NVIDIA GPUs
• Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and handling thousands of concurrent connections reliably
• PhD in CS, Physics, or Math, or equivalent practical experience building backend or ML systems
• Ability to learn quickly, thrive in ambiguity, choose a direction, and work independently
• Focus on shipping stable solutions and delivering impact
• Curiosity about the reasoning behind technical decisions and willingness to challenge architectures
• Будет плюсом: Non-trivial systems programming projects, open-source contributions to major inference engines, deep-dive technical write-ups

• Base salary range: £140,000 – £200,000
• Total compensation includes equity and benefits
• Individual pay within the range depends on work location, level, competencies, experience, and business needs
• Candidates must already have the legal right to work in the United Kingdom; visa sponsorship is not available
• Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area in the future, subject to business needs and applicable legal and work authorization requirements