Software Engineer, LLM Inference
Удалённо · Полная занятость
An engineering role focused on optimizing a vLLM-based LLM inference stack for low latency and high GPU utilization. Requires deep hands-on experience with inference systems, CUDA/Triton kernels, and distributed GPU execution. Remote or Slovenia-based, working on a voice AI platform.
About the Role Our client is pushing the boundaries of real-time AI, and we're looking for an engineer to help us run large language models with exceptional speed, efficiency, and reliability at production scale.
In this role, you'll work deep in the LLM inference stack, from vLLM scheduling and KV-cache management to CUDA kernels, distributed execution, and GPU profiling, optimizing every part of the path from request to generated token.
Responsibilities
In this role, you will:
- Build and optimize our vLLM-based inference stack for low latency, high throughput, and maximum GPU utilization.
- Optimize continuous batching, scheduling, prefill/decode, prefix caching, and KV-cache allocation and reuse.
- Optimize or implement CUDA and Triton kernels for attention, GEMMs, sampling, normalization, and other critical model operations.
- Evaluate and integrate technologies such as FlashAttention, FlashInfer, CUDA Graphs, torch.compile, speculative decoding, and quantization.
- Optimize distributed inference using tensor, data, and expert parallelism, NCCL, NVLink/NVSwitch, and InfiniBand.
- Work closely with researchers to bring new dense and MoE model architectures into production quickly and efficiently.
Requirements & Qualifications
You might thrive in this role if you:
- Have deep hands-on experience with LLM inference, ideally working inside vLLM, SGLang, TensorRT-LLM, or similar systems, not just deploying them.
- Understand TTFT, inter-token latency, throughput, continuous batching, PagedAttention, KV caching, and prefill vs. decode performance.
- Are comfortable profiling GPUs and reasoning about compute, memory bandwidth, kernel launches, synchronization, and communication bottlenecks.
- Have experience with CUDA, Triton, PyTorch, NCCL, and modern NVIDIA GPU architectures.
- Understand Transformer internals including MHA/GQA, ROPE, KV cache, quantization, and MoE.
- Have experience optimizing distributed, performance-critical systems in production.
- Care deeply about performance, simplicity, and reliability, and take ownership from profiling through production deployment.
Why Us
- You'll help build one of the most technically advanced voice AI platforms in the world, and push LLM inference performance at every layer of the stack.
- You'll work directly with a world-class team of engineers and researchers on hard, measurable problems spanning models, GPU kernels, distributed systems, and production infrastructure.
- You'll have a voice in how our technology evolves, how our company grows, and how AI transforms human communication.
Навыки
Похожие вакансии
Senior QA Engineer
FxPro
Senior QA Engineer в FxPro на Кипре. Помогают с релокацией.
Technical Account Manager, AWS Enterprise Support
от €3 683 в месяцAWS · Офис
AWS ищет Technical Account Manager в команду Enterprise Support — доверенного технического советника для корпоративных клиентов на Кипре и в регионе. Требуются опыт траблшутинга минимум в 3 технических областях, инженерный бэкграунд, свободный русский или украинский и уверенный английский. Юридически позиция в Греции, но сотрудник должен физически находиться на Кипре.
DevOps-инженер (Middle / Middle+)
Удалённо
Удалённая позиция DevOps-инженера в международной продуктовой компании, разрабатывающей профессиональное электронное оборудование и софт для геодезистов, строителей и картографов. Нужен практический опыт с Linux, Docker, Kubernetes, Git, CI/CD и мониторингом, автоматизация на Python/Bash/Go, профильное высшее образование и английский B1+. Без on-call, оплата в евро, 20 дней отпуска и оплачиваемый больничный.
QA-инженер (AI First, CRM)
$2 500–3 500 в месяцPrionex · Удалённо
Международная IT-компания Prionex, развивающая CRM-платформу, ищет QA-инженера в формате AI First. Задачи: тестирование новых возможностей CRM от требований до проверки после релиза, разработка API/интеграционных/E2E-автотестов в CI/CD, использование AI-агентов как основного рабочего инструмента. Требуется опыт тестирования веб-приложений 1–3 года, автоматизация на Python или TypeScript/JS, Playwright или аналоги, практический опыт с AI-агентами.
IT / Системный администратор
Роль в IT и системном администрировании, охватывающая инфраструктуру и контроль операций, Кипр — одна из локаций. Предлагаются релокационная поддержка и премиальные бонусы.
Data Engineer (Python/Pandas/Snowflake/AWS)
$5 700–6 600 в месяцİndustrial · Удалённо
Remote B2C contract data engineering role for a Chicago-based company. Requires 8+ years of data engineering and Python experience, strong SQL, Snowflake dashboards, PySpark/Pandas, AI/agentic frameworks and AWS. Contract minimum 9 months, 4h daily overlap with US hours, $5700-6600 before taxes.