LetsGo Jobs

Software Engineer, LLM Inference

Remote · Full-time

An engineering role focused on optimizing a vLLM-based LLM inference stack for low latency and high GPU utilization. Requires deep hands-on experience with inference systems, CUDA/Triton kernels, and distributed GPU execution. Remote or Slovenia-based, working on a voice AI platform.

About the Role Our client is pushing the boundaries of real-time AI, and we're looking for an engineer to help us run large language models with exceptional speed, efficiency, and reliability at production scale.

In this role, you'll work deep in the LLM inference stack, from vLLM scheduling and KV-cache management to CUDA kernels, distributed execution, and GPU profiling, optimizing every part of the path from request to generated token.

Responsibilities

In this role, you will:

  • Build and optimize our vLLM-based inference stack for low latency, high throughput, and maximum GPU utilization.
  • Optimize continuous batching, scheduling, prefill/decode, prefix caching, and KV-cache allocation and reuse.
  • Optimize or implement CUDA and Triton kernels for attention, GEMMs, sampling, normalization, and other critical model operations.
  • Evaluate and integrate technologies such as FlashAttention, FlashInfer, CUDA Graphs, torch.compile, speculative decoding, and quantization.
  • Optimize distributed inference using tensor, data, and expert parallelism, NCCL, NVLink/NVSwitch, and InfiniBand.
  • Work closely with researchers to bring new dense and MoE model architectures into production quickly and efficiently.

Requirements & Qualifications

You might thrive in this role if you:

  • Have deep hands-on experience with LLM inference, ideally working inside vLLM, SGLang, TensorRT-LLM, or similar systems, not just deploying them.
  • Understand TTFT, inter-token latency, throughput, continuous batching, PagedAttention, KV caching, and prefill vs. decode performance.
  • Are comfortable profiling GPUs and reasoning about compute, memory bandwidth, kernel launches, synchronization, and communication bottlenecks.
  • Have experience with CUDA, Triton, PyTorch, NCCL, and modern NVIDIA GPU architectures.
  • Understand Transformer internals including MHA/GQA, ROPE, KV cache, quantization, and MoE.
  • Have experience optimizing distributed, performance-critical systems in production.
  • Care deeply about performance, simplicity, and reliability, and take ownership from profiling through production deployment.

Why Us

  • You'll help build one of the most technically advanced voice AI platforms in the world, and push LLM inference performance at every layer of the stack.
  • You'll work directly with a world-class team of engineers and researchers on hard, measurable problems spanning models, GPU kernels, distributed systems, and production infrastructure.
  • You'll have a voice in how our technology evolves, how our company grows, and how AI transforms human communication.

Skills

LLM inferencevLLMCUDATritonPyTorchNCCLGPU profilingdistributed systemsquantizationKV caching

Similar vacancies

QA Engineer (AI First, CRM)

$2,500–3,500 per month

Prionex · Remote

Prionex, an international IT company building a CRM platform, is hiring a QA Engineer to work AI-first. The role covers testing new CRM features from requirements to post-release checks, building API/integration/E2E automated tests in CI/CD, and using AI agents as the main working tool. Requires 1-3 years of web testing experience, Python or TypeScript/JS automation, Playwright or similar, and hands-on AI-agent experience.

IT & developmentQAtest automationPython
21 November

IT / System Administrator

IT and system administration role covering infrastructure and operations control, with Cyprus among available locations. Relocation support and premium perks are offered.

IT & developmentsystem administrationinfrastructureoperations
21 November

Product Manager (Billing)

Quadcode · Remote

Quadcode, a fintech company selling its trading platform as SaaS to brokers, is looking for a Product Manager (Billing) to own the entire flow of customer funds, from deposits to withdrawals. Candidates need 3+ years in billing and payments, knowledge of deposit/withdrawal mechanics, routing and cascading, high-risk experience (e.g. Forex), KYC/AML rules, payment metrics, SQL and A/B testing, plus native Russian and English B2+. The job is remote with flexible hours, direct product influence, creative freedom, AI tooling and growth opportunities.

IT & developmentBilling & paymentsPayment provider integrationDeposit/withdrawal mechanics
21 November

Product Manager (Crypto)

PayDepot

PayDepot, a major Bitcoin ATM network, is hiring a Product Manager to drive growth for its non-custodial wallet and crypto card. Requires 3+ years of product management in fintech or Web3, blockchain knowledge, and familiarity with EVM standards. Relocation support is provided.

IT & developmentProduct ManagementFintechWeb3
21 November

Senior iOS Engineer

Quadcode

Senior iOS engineer role at a fintech company in Cyprus. Details available via the application link.

IT & developmentiOS developmentSwift
21 November

Head of R&D (AI-first)

Quadcode

Lead AI-driven automation across distributed teams at a fintech company, improving development workflows and engineering practices. Requires 5+ years in software development leadership, 3+ years as Head of R&D or Engineering Director, strong backend experience, and proven AI implementation in development workflows. Fluent Russian and B2 English required.

IT & developmentsoftware development leadershipR&D managementbackend development
21 November