Institute of Foundation Models

Researches and develops foundation models

Research Engineer/Scientist - Speech/Audio Machine Learning

AI Research +1Confirmed live in the last 24 hours

Full-timeParis, FranceNo salary listedIn-person

About the Job

Requirements

A PhD or MSc in Computer Science with a focus on Deep Learning, Signal Processing, or Computational Linguistics.
A record of research demonstrated through published work in top-tier venues including NeurIPS, ICLR, ICASSP, or Interspeech.

Responsibilities

Design and train state-of-the-art end-to-end neural speech models.
Develop novel neural architectures for low-latency speech-to-speech translation and generation, including diffusion, flow-matching, and transformer-based audio large language models.
Design and implement custom objective functions to optimize prosody, including emotions, intelligibility, and naturalness.
Conduct large-scale training runs and perform ablation studies on model architecture and tokenization strategies.
Establish rigorous internal benchmarks using objective metrics such as word error rate and mel-cepstral distortion, along with subjective human-in-the-loop mean opinion score testing.
Develop core intellectual property and advance native audio-to-audio multimodal architectures beyond cascaded automatic speech recognition to text-to-speech systems.

About the Institute of Foundation Models

We are a dedicated research lab for building, understanding, using, and risk-managing foundation models. Our mandate is to advance research, nurture the next generation of AI builders, and drive transformative contributions to a knowledge-driven economy.

As part of our team, you’ll have the opportunity to work on the core of cutting-edge foundation model training, alongside world-class researchers, data scientists, and engineers, tackling the most fundamental and impactful challenges in AI development. You will participate in the development of groundbreaking AI solutions that have the potential to reshape entire industries. Strategic and innovative problem-solving skills will be instrumental in establishing MBZUAI as a global hub for high-performance computing in deep learning, driving impactful discoveries that inspire the next generation of AI pioneers.

The Role

As a Research Scientist specializing in speech/audio machine learning, you will contribute to the design and training of SoTA end-to-end neural speech models. You will be responsible for developing the core intellectual property, moving beyond cascaded ASR → TTS systems toward native audio-to-audio multimodal architectures.

Key responsibilities

Architectural Design: Develop novel neural architectures for low-latency speech-to-speech translation and generation (e.g., Diffusion, Flow-matching, Transformer-based audio LLMs).
Loss Function Engineering: Design and implement custom objective functions to optimize prosody (emotions, intelligibility, naturalness).
Experimental Iteration: Conduct large-scale training runs, performing ablation studies on model architecture and tokenization strategies.
Evaluation Frameworks: Establish rigorous internal benchmarks using both objective metrics (WER, MCD) and subjective human-in-the-loop (MOS) testing.

Qualifications

PhD or MSc in Computer Science with a focus on Deep Learning, Signal Processing, or Computational Linguistics.
Record of Research: Published work in top-tier venues (NeurIPS, ICLR, ICASSP, Interspeech).

About the Company

Institute of Foundation Models

Researches and develops foundation models

IFM builds and studies foundation models—large AI models designed to learn broadly from diverse data and be used across many tasks. Its work centers on academic research to create open, fast, and practical models that address real-world societal needs, rather than narrow applications. The models are developed by a global team across Abu Dhabi, Paris, and Silicon Valley, with a strong emphasis on openness and collaboration to advance AI science and accessibility. Unlike typical private labs that lock models behind paywalls, IFM aims to provide publicly accessible, efficient models that can be used by researchers and developers to solve real problems. The overarching goal is to push the science of foundation models forward while ensuring their benefits reach society at large.

Company size: Not listed
Company stage: Not listed
Total funding: Not listed
Headquarters: United Arab Emirates
Founded: Not listed

Recently Posted Jobs

Explore more opportunities at Institute of Foundation Models

See more jobs