K

KRAFTON

Develops and publishes cross-platform games

AI Inference Platform & Operations Lead

Platform Engineering +1Updated on Nov 7
Full-timeSeoul, South KoreaNo salary listed

About the Job

Requirements

  • At least 10 years of experience in infrastructure, site reliability engineering, or platform engineering for large-scale traffic services, and experience leading a team responsible for critical-path production systems.
  • Ability to compare and select core technologies such as serving engines, orchestration layers, and storage layers, and explain trade-offs among performance, cost, and operational complexity with supporting evidence.
  • Ability to personally design and review important technical bottlenecks and lead the team's execution.
  • Experience defining latency, availability, throughput, and cost metrics, making decisions among competing objectives, and achieving measurable improvements.
  • Ability to negotiate service-level objectives and costs numerically with business teams and executives, and document inter-team responsibilities and technical decisions clearly.
  • Experience designing and operating deployment, load balancing, scheduling, autoscaling, observability, on-call, and incident-response systems in large-scale distributed systems using Kubernetes or equivalent orchestration.
  • Experience deploying and operating LLM or machine-learning model serving, or equivalent high-compute, low-latency real-time systems such as real-time 3D games, in production GPU environments.
  • Experience operating capacity planning, quota and priority, and resource-isolation policies where demand changes rapidly and resources are constrained.
  • Experience writing production service code in Python or Go and reading and modifying code in large open-source systems.
  • No impediment to international business travel.

Responsibilities

  • Own the inference fleet and control plane, including the platform's technical direction and major technical decisions; design and operate a serving platform for proprietary and open-weight models, including voice and multimodal models, and establish its operating processes while leading team execution.
  • Select serving engines and orchestration layers, and establish the technical roadmap and operational standards for request routing, capacity, model and cache placement, autoscaling, and multi-model and multi-tenant operating policies.
  • Build and develop the team through direct hiring, and serve as the primary point of contact among AI Research's compute-optimization R&D organization, the internal IT organization operating inference services, and business squads.
  • Translate production workload profiles and bottlenecks into R&D requirements, and create a loop for validating, accepting, and deploying research results.
  • Build performance and quality regression testing for new models, release automation, canary deployment and rollback, API gateway authentication, quotas and rate limits, and token-level observability.
  • Provide self-service tools and guidance that enable business squads and model developers to onboard models quickly and consistently.
  • Define workload-specific service-level objectives and error budgets with each squad for time to first token, time per output token, end-to-end p95 and p99 latency, availability, error rate, and throughput.
  • Establish operating practices covering on-call, incident response, postmortems, capacity events, and deployment safety.
  • Lead architecture improvements based on postmortems.
  • Design GPU scheduling and allocation, tenant isolation, quota and priority policies, and capacity-planning models.
  • Measure GPU utilization, throughput, and per-request, per-token, and per-work-unit costs by model and workload, and provide metrics that let squads plan demand and cost.
  • Set strategies for applying continuous batching, prefix caching, key-value cache offloading and tiering, prefill-decode disaggregation, request scheduling, and model routing to real traffic, and lead execution.
  • Improve routing, scheduling, caching logic, and model-integration code needed by the service directly with the team.
  • Establish validation criteria for quality regressions, performance effects, and failure scope for engine or R&D features such as quantized checkpoints and speculative decoding, and lead staged rollouts.
  • Provide session-based routing, cache reuse, structured output, and small-to-large model cascading for agent workloads with long contexts and repeated tool calls, applying them with business squads.
  • For voice services, own the performance and reliability of the STT–LLM–TTS inference stages and common serving capabilities against latency and quality targets agreed with squads; establish observability criteria for end-to-end latency, including turn-taking, and lead improvements.

Desired Qualifications

  • Experience operating LLM serving engines such as vLLM, SGLang, TensorRT-LLM, or Triton Inference Server in production.
  • Experience operating distributed inference orchestration or inference control planes such as NVIDIA Dynamo, llm-d, KServe, or Ray Serve.
  • Experience introducing serving features such as continuous batching, prefill-decode disaggregation, prefix caching, and key-value cache offloading or tiering to real traffic and measuring their impact.
  • Experience applying optimization features such as quantized checkpoints and speculative decoding in production with accuracy regression tests, canary deployments, and rollback.
  • Experience operating multi-node GPU environments using NVLink or InfiniBand/RoCE, and GPU resource-management tools such as GPU Operator, DCGM, or MIG.
  • Experience serving real-time streaming voice conversation systems combining speech-to-text, large language models, text-to-speech, and a turn detector, or voice, video, or multimodal models.
  • Experience optimizing low-latency WebSocket or server-sent events pipelines, including end-to-end time to first token, turn-taking, and backpressure.
  • Experience serving agent workloads with frequent tool-call loops, such as coding agents, or operating an LLM gateway such as LiteLLM and a model-routing layer.
  • Experience building a new team or platform organization from the ground up, including hiring and development.
  • Experience leading the production integration of researcher-developed models or algorithms, including interface agreements, validation loops, regression testing, and deployment.
  • Experience providing models to external customers through APIs and operating service-level agreements, quotas, and billing.
  • Experience building GPU cost-visibility systems such as FinOps or chargeback/showback.
  • Experience forking large open-source systems and synchronizing upstream changes, or operating release and branch strategies for B2B multi-tenant environments.
  • Experience operating serving on non-NVIDIA accelerators such as NPUs.

About the Company

KRAFTON

Develops and publishes cross-platform games

Krafton is a group of independent game studios that develops and publishes video games for PC, console, and mobile. Its business model centers on delivering high-quality games and monetizing them through game sales, in-game purchases, and licensing deals, with PUBG as a flagship title. Krafton differentiates itself by coordinating a network of studios and investing in technology and publishing support rather than relying on a single in-house studio, enabling a diverse catalog and esports initiatives. Its goal is to push the boundaries of gaming and connect people through shared entertainment moments across platforms.

Company size
51-200
Company stage
IPO
Headquarters
Seongnam-si, South Korea
Founded
2007

Recently Posted Jobs

Explore more opportunities at KRAFTON

See more jobs