описание
The Social Health Insurance Fund is a national organization based in Astana. It is building a nationwide AI platform for healthcare on in-house infrastructure equipped with NVIDIA H200 GPUs. The platform is being developed from the ground up, offering an opportunity to influence its architecture.
задачи
- Deploy LLMs, embedding models, and reranking models in a closed environment and ensure stable inference
- Optimize inference speed and GPU usage through batching, KV caching, and quantization
- Develop infrastructure for RAG and AI agents in collaboration with AI and Data Engineers
- Establish CI/CD, model versioning and rollback, monitoring, access controls, and auditing
требования
- Higher education in IT
- At least 3 years of experience, including production operation of ML/LLM services
- Experience with Linux, Docker, Kubernetes, and Helm
- Ability to deploy LLMs using vLLM, SGLang, or Triton, with an understanding of inference systems
- Experience with GPU infrastructure, NVIDIA drivers, and CUDA
- Python and Bash skills, CI/CD experience, and familiarity with Prometheus and Grafana
условия
- Astana / Office
- 1,500,000 to 2,000,000 KZT Gross per month