A

Aleroy

Unified API for scalable AI inference

Staff / Senior Machine Learning Engineer - Post-Training & Inference

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-time

About the Job

Requirements

  • At least five years of experience writing production software.
  • Fluency in Python and PyTorch.
  • Hands-on experience post-training large language models for real use, including supervised fine-tuning and LoRA, plus at least one of DPO, ORPO, reinforcement learning (GRPO, PPO, or similar), or distillation.
  • Experience building an evaluation set or reward function from real usage data and demonstrating that a fine-tuned model outperformed its baseline.
  • Experience training across several GPUs with FSDP, DeepSpeed, or a similar tool.
  • Experience shipping a trained model to production using vLLM, SGLang, TensorRT-LLM, or a similar system, measuring its performance, and resolving issues after release.
  • Advanced English proficiency (C1).

Responsibilities

  • Post-train open models for customers, using methods such as LoRA or full-parameter training, supervised fine-tuning, DPO and ORPO, reinforcement learning with custom rewards, and distillation into smaller models.
  • Choose the post-training method for each customer based on evaluation results.
  • Enable customers to bring their own losses, rewards, and rollouts and run them on the training stack.
  • Turn customers’ production traces and product signals into evaluation sets and reward functions.
  • Run scheduled training and validation against the production model, ship updates only when they perform better, and roll back regressions.
  • Keep training and serving numerically consistent so the evaluated model matches the served model.
  • Promote trained adapters or models to production in one step, including multiple adapters on shared capacity and quantised base models.
  • Train draft models on each customer’s traffic for speculative decoding.
  • Meet each customer’s latency target through throughput optimization, sharding, prefill, decode and cache layout, batching, and FP8 or FP4, with quality checks that include Arabic.
  • Automatically tune deployments by having agents propose serving changes, including sharding, batching, draft models, and precision, and ship changes only when they are faster and remain correct on the customer’s evaluation.
  • Reduce cost at equal quality by routing and cascading across models, measured on the customer’s evaluation.
  • Own model customization end to end and contribute hands-on code each week.
  • Work with the Tech Lead and CTO.

Desired Qualifications

  • For the Staff level, experience leading a technical project lasting several months across multiple engineers and making trade-offs.

About the Company

Aleroy

Unified API for scalable AI inference

Aleroy provides a single API for running AI models at scale, supporting both ready-to-use and custom models across cloud, dedicated capacity, and customer-owned hardware. It handles the entire inference lifecycle—optimization, deployment, and scaling—while letting enterprises choose where workloads run and ensuring data remains in their own infrastructure, with a zero-retention policy on prompts and outputs. The platform supports both dedicated and shared inference and differentiates itself with infrastructure-agnostic delivery via one API and options to monetize idle GPU capacity through white-labeled services or GPU-sharing partnerships. Its goal is to give enterprises precise control over AI deployments, optimize performance and cost, and enable secure, scalable inference across multiple environments.

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
Not listed
Founded
Not listed

Recently Posted Jobs

Explore more opportunities at Aleroy

See more jobs