A

Aleph Alpha

Full-stack generative AI for sovereign enterprises

Senior AI Researcher-Pre-training - f/m/d

AI ResearchUpdated on Nov 16
Full-timeNo salary listed

About the Job

Requirements

  • Proficiency in Python and deep familiarity with PyTorch-based training workflows.
  • A strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published research.
  • A strong mathematical foundation and comfort reasoning formally about optimization, scaling behavior, and training dynamics.
  • Deep understanding of transformer training dynamics, optimization, and the behavior of large distributed training jobs.
  • Ability to design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.
  • Hands-on experience pre-training large models, such as models with 7B or more parameters, on substantial infrastructure such as clusters with 100 or more GPUs.
  • Strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation workflows.
  • Ability to implement complex model architectures efficiently and reliably and to debug complex issues across model code, training dynamics, and distributed systems.
  • Ability to work in Germany and collaborate regularly on site in Heidelberg as part of the Pre-training team.

Responsibilities

  • Own core elements of the training recipe, including optimizers, schedules, and initialization, and design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.
  • Develop hyperparameter scaling laws and scale-up methodologies using small-scale proxy experiments to predict multi-thousand-GPU behavior and de-risk major training decisions.
  • Investigate complex convergence issues, including loss spikes and divergence, and resolve difficult-to-reproduce distributed-system failures such as communication bottlenecks, race conditions, and synchronization errors.
  • Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.

Desired Qualifications

  • Hands-on experience training large language models or multimodal models on large GPU clusters using distributed frameworks such as Megatron-LM, DeepSpeed, or torchtitan.
  • Familiarity with scaling laws, hyperparameter transfer, or methods for predicting large-scale training behavior from smaller proxy runs.
  • Experience profiling distributed jobs and diagnosing training anomalies such as loss spikes, numerical instability, or optimizer pathologies.
  • Exposure to sparse training approaches such as Mixture-of-Experts and an understanding of their routing and systems trade-offs.
  • Demonstrated research excellence through top-tier publications such as NeurIPS, ICML, or ICLR, impactful open-source contributions, or significant shipped technical work.
  • Strong curiosity about hardware and systems constraints that shape scale; low-level kernel optimization is not required.

About the Company

Aleph Alpha

Full-stack generative AI for sovereign enterprises

Aleph Alpha builds sovereign, human-centric artificial intelligence in Europe. It offers PhariaAI, a full-stack generative AI suite that lets enterprises and governments design and manage bespoke AI use cases, delivered as PhariaAI-as-a-Service. The product works by providing a modular platform where customers specify their AI needs, deploy models, and govern usage within secure, policy-driven environments—often through partnerships with other firms to reach more clients. Compared with global players, Aleph Alpha differentiates itself by focusing on European sovereignty and trust, tailoring solutions for security-critical applications, and operating through a collaboration-based business model rather than a single product sale. Its goal is to provide trustworthy, enterprise-grade AI for government and industry, offering a European alternative to dominant AI providers.

Company size
201-500
Company stage
Series B
Total funding
$641.1M
Headquarters
Heidelberg, Germany
Founded
2019

Recently Posted Jobs

Explore more opportunities at Aleph Alpha

See more jobs