Software Engineer, LLM Inference
Remote · Vollzeit
An engineering role focused on optimizing a vLLM-based LLM inference stack for low latency and high GPU utilization. Requires deep hands-on experience with inference systems, CUDA/Triton kernels, and distributed GPU execution. Remote or Slovenia-based, working on a voice AI platform.
About the Role Our client is pushing the boundaries of real-time AI, and we're looking for an engineer to help us run large language models with exceptional speed, efficiency, and reliability at production scale.
In this role, you'll work deep in the LLM inference stack, from vLLM scheduling and KV-cache management to CUDA kernels, distributed execution, and GPU profiling, optimizing every part of the path from request to generated token.
Responsibilities
In this role, you will:
- Build and optimize our vLLM-based inference stack for low latency, high throughput, and maximum GPU utilization.
- Optimize continuous batching, scheduling, prefill/decode, prefix caching, and KV-cache allocation and reuse.
- Optimize or implement CUDA and Triton kernels for attention, GEMMs, sampling, normalization, and other critical model operations.
- Evaluate and integrate technologies such as FlashAttention, FlashInfer, CUDA Graphs, torch.compile, speculative decoding, and quantization.
- Optimize distributed inference using tensor, data, and expert parallelism, NCCL, NVLink/NVSwitch, and InfiniBand.
- Work closely with researchers to bring new dense and MoE model architectures into production quickly and efficiently.
Requirements & Qualifications
You might thrive in this role if you:
- Have deep hands-on experience with LLM inference, ideally working inside vLLM, SGLang, TensorRT-LLM, or similar systems, not just deploying them.
- Understand TTFT, inter-token latency, throughput, continuous batching, PagedAttention, KV caching, and prefill vs. decode performance.
- Are comfortable profiling GPUs and reasoning about compute, memory bandwidth, kernel launches, synchronization, and communication bottlenecks.
- Have experience with CUDA, Triton, PyTorch, NCCL, and modern NVIDIA GPU architectures.
- Understand Transformer internals including MHA/GQA, ROPE, KV cache, quantization, and MoE.
- Have experience optimizing distributed, performance-critical systems in production.
- Care deeply about performance, simplicity, and reliability, and take ownership from profiling through production deployment.
Why Us
- You'll help build one of the most technically advanced voice AI platforms in the world, and push LLM inference performance at every layer of the stack.
- You'll work directly with a world-class team of engineers and researchers on hard, measurable problems spanning models, GPU kernels, distributed systems, and production infrastructure.
- You'll have a voice in how our technology evolves, how our company grows, and how AI transforms human communication.
Skills
Ähnliche Stellen
Senior QA Engineer
FxPro
Senior QA Engineer at FxPro in Cyprus. Relocation support is provided.
Technical Account Manager, AWS Enterprise Support
ab €3.683 pro MonatAWS · Büro
AWS is hiring a Technical Account Manager for its Enterprise Support team, acting as a trusted technical advisor to enterprise customers in Cyprus and the surrounding region. Candidates need troubleshooting experience in at least three technical domains, an engineering background, fluent Russian or Ukrainian, and strong English. The role is legally based in Greece but the employee must physically be in Cyprus.
DevOps Engineer (Middle / Middle+)
Remote
Remote DevOps role at an international product company building professional electronic equipment and software for surveyors, builders and cartographers. Requires hands-on Linux, Docker, Kubernetes, Git, CI/CD and monitoring experience plus scripting in Python, Bash or Go, an engineering degree and English at B1+. No on-call, euro pay, 20 vacation days and paid sick leave.
QA Engineer (AI First, CRM)
$2.500–3.500 pro MonatPrionex · Remote
Prionex, an international IT company building a CRM platform, is hiring a QA Engineer to work AI-first. The role covers testing new CRM features from requirements to post-release checks, building API/integration/E2E automated tests in CI/CD, and using AI agents as the main working tool. Requires 1-3 years of web testing experience, Python or TypeScript/JS automation, Playwright or similar, and hands-on AI-agent experience.
IT / System Administrator
IT and system administration role covering infrastructure and operations control, with Cyprus among available locations. Relocation support and premium perks are offered.
Data Engineer (Python/Pandas/Snowflake/AWS)
$5.700–6.600 pro Monatİndustrial · Remote
Remote B2C contract data engineering role for a Chicago-based company. Requires 8+ years of data engineering and Python experience, strong SQL, Snowflake dashboards, PySpark/Pandas, AI/agentic frameworks and AWS. Contract minimum 9 months, 4h daily overlap with US hours, $5700-6600 before taxes.