V

Veeda AI

High-fidelity simulated environments for Physical AI

Member of Technical Staff - ML Performance

Applied Machine LearningConfirmed live in the last 24 hours
Full-timeNo salary listedHybrid

About the Job

Requirements

  • A bachelor's degree or equivalent hands-on experience in Computer Science, Computer Engineering, or a related technical field is required.
  • Deep hands-on experience with PyTorch and at least one large-scale parallelism stack (FSDP2, Megatron-Core, TorchTitan, or DeepSpeed) on real multi-node jobs is required.
  • Fluency in Python and C++/CUDA, including the ability to predict kernel stalls from memory access patterns before profiling, is required.
  • Experience profiling live training and inference runs with Nsight Systems or the PyTorch profiler and translating traces into quantifiable step-time or model FLOPs utilization improvements is required.
  • Expertise in at least one of low-precision numerics, kernel authoring, or large-run fault diagnosis, and credibility in the others, is required.

Responsibilities

  • Own step time and model FLOPs utilization for multi-node video world model training, choosing the tensor, context, and expert parallelism mix in PyTorch FSDP2 and Megatron-Core.
  • Profile end-to-end latency, GPU utilization, memory pressure, and kernel efficiency with PyTorch Profiler, Nsight Systems, Nsight Compute, and torch.utils.benchmark, then improve throughput through torch.compile/Inductor, CUDA Graphs, mixed and low precision, quantization, operator fusion, and multi-GPU serving.
  • Take BF16, FP8, and NVFP4 recipes from running to converging on Blackwell, tracing scaling-factor and accumulation bugs into video tokenizer and VAE layers where activation outliers occur.
  • Write and tune CUDA and Triton kernels, driving FlashAttention-4, FlexAttention, and torch.compile integration to address quadratic attention over long video sequences.
  • Tune NCCL collectives and compute/communication overlap across NVLink domains and the fabric, using the NCCL flight recorder to identify watchdog timeouts by rank and collective.
  • Build detection for silent data corruption, stuck CUDA kernels, and card-freeze hangs, plus asynchronous and tiered checkpointing to reduce interruption recovery time.

Desired Qualifications

  • Experience optimizing training or inference workloads across very large GPU clusters, including topology-aware placement, scaling efficiency, performance isolation, and diagnosing failures that emerge only at fleet scale.
  • Experience writing Triton, CUTLASS, or CuTe-DSL kernels, or contributing to open-source kernel libraries.
  • Experience implementing context or sequence parallelism for long-horizon video or high-token-count models.
  • Experience running or porting large training workloads on AMD GPUs using ROCm or Google TPUs using JAX/XLA.
  • Experience building fault-tolerant training with elastic world size, dynamic node re-queueing, or asynchronous distributed checkpointing.
  • Experience optimizing generative inference for interactive rollouts, including few-step samplers, distillation, and key-value or latent caching.
  • Publications or presentations on machine learning systems, compilers, or high-performance kernels.

About the Company

Veeda AI

High-fidelity simulated environments for Physical AI

Veeda AI builds high-fidelity, scalable simulated environments powered by world models for Physical AI and robotics. It creates an internal multimodal simulation of the real world that embodied agents can learn from through safe virtual interaction, avoiding real-world risks. The company differentiates itself with an Nvidia-inspired research lineage and a focus on large, general-purpose world models plus a scalable AI simulation layer that supports the entire Physical AI ecosystem. Its goal is to remove learning and deployment bottlenecks to accelerate progress in AI, robotics, and embodied agents.

Company size
1-10
Company stage
Seed
Total funding
$90M
Headquarters
Toronto, Canada
Founded
2026

Recently Posted Jobs

Explore more opportunities at Veeda AI

See more jobs