W

White Circle

Production AI governance and monitoring platform

Multimodal ML Engineer

Natural Language Processing & LLMsUpdated on Nov 6
Full-time

About the Job

Requirements

  • At least 3 years of training large-scale deep learning models in multimodal domains such as vision-language, audio, speech, or acoustics.
  • Strong PyTorch skills and hands-on distributed training experience with DeepSpeed, FSDP, or similar tools.
  • Deep experience with multimodal architectures, including how vision and audio encoders, projectors, and large language models work together; examples include LLaVA, Qwen-VL, InternVL, Audio Flamingo, Omni Qwen, Audio Qwen, Whisper, HuBERT, and Conformer.
  • Hands-on experience with multimodal reinforcement learning from human feedback and alignment, including GRPO, DPO, or reward modeling.
  • Experience with video or audio sequence modeling, including temporal modeling, long-context processing, efficient attention, or streaming inference.
  • A track record of shipping models to production, meeting latency targets, and optimizing inference.
  • Experience curating large-scale multimodal datasets, including image-text pairs, video-instruction data, audio preprocessing, augmentation, and synthetic data generation.
  • Familiarity with mixture-of-experts architectures and their trade-offs for multimodal workloads.
  • Strong engineering fundamentals, including clean code, version control, testing, and documentation.

Responsibilities

  • Train and fine-tune large-scale multimodal models for vision-language, audio, and speech from scratch and from pretrained checkpoints.
  • Extend models across modalities, including image understanding, video temporal modeling, long-context processing, and streaming audio.
  • Design and run experiments involving architecture changes, data mixes, and training recipes.
  • Build and maintain multimodal data pipelines that turn raw images, video, and audio recordings into training-ready datasets, including synthetic data generation.
  • Train and optimize mixture-of-experts architectures for efficient multimodal inference.
  • Build cross-modal alignment pipelines using supervised fine-tuning, direct preference optimization, group relative policy optimization, and reward modeling.
  • Optimize models for production through quantization, distillation, batching, streaming, and low-latency serving.
  • Deploy models end-to-end, from research checkpoint to production serving.
  • Define product-relevant evaluation metrics and benchmarks for visual question answering, spatial reasoning, video comprehension, and speech and audio understanding.

Desired Qualifications

  • Understanding of audio signal processing fundamentals, including spectrograms, mel features, and noise reduction.

About the Company

White Circle

Production AI governance and monitoring platform

White Circle provides an enterprise AI governance layer via a single API that tests, protects, observes, and improves production AI systems in real time. Its models monitor inputs and outputs against policies to detect harmful content, hallucinations, prompt injections, jailbreaks, model drift, and misuse, with analytics to boost reliability. It differentiates itself with end-to-end production oversight, real-time monitoring, policy-driven checks across inputs and outputs, and a strong compliance posture for clients like large digital banks. Its goal is to help organizations run safe, reliable, and compliant AI at scale.

Company size
11-50
Company stage
Seed
Total funding
$11M
Headquarters
Paris, France
Founded
2025

Recently Posted Jobs

Explore more opportunities at White Circle

See more jobs