Technical Account Manager - AI Infrastructure

Prime Intellect's IPO stops AI training crashes that derailed long runs. Prime Intellect ships a new default RL loss built for async training, plus a 1.15x throughput bump over torchtitan on Qwen3-235B. · 3 hrs ago Read 5 min * New default RL loss IPO masks on probability difference instead of ratio, stabilizing async training. * In SWE RL tests, IcePop collapsed at step 1.2k while IPO trained through, masking just 0.25% of tokens. * Qwen3-235B SFT runs 1.15x faster than torchtitan on identical 4x8 B300 nodes at 46.6% MFU. * GLM-5.3 sparse attention via FlashMLA + cuDNN pushes MFU from 22.3% to 40.1% on 12 nodes. * Native support added for DeepSeek V4 Flash, Qwen3.8 Flash Next, Nemotron-H, and multimodal RL (Qwen-VL, Kimi K2.5). * Unified uv run eval CLI replaces separate eval tooling; verifiers v0.4.0 removes the legacy v0 stack. Prime Intellect has released prime-rl v0.10.0 and verifiers v0.4.0. The central change is INTELLECT Policy Optimization, or IPO, a new default reinforcement-learning loss designed to stabilize asynchronous training when the trainer and inference engine assign different probabilities to the same tokens. The release also expands native model support, consolidates evaluation tooling, and improves reported mixture-of-experts training throughput. Trainer drift can derail long runs. Asynchronous reinforcement learning generates rollouts, the model's sampled responses, in an inference engine such as vLLM while a separate trainer updates the model. A rollout may reach the trainer after several updates, so it reflects an older policy. Different GPU kernels in the inference and training stacks can introduce additional numerical differences. The resulting trainer-inference drift changes the probability assigned to each sampled token. Large discrepancies make the training signal unreliable and can destabilize long-running, off-policy workloads. IcePop, a loss previously used to control this drift, masks tokens when the ratio between trainer and inference probabilities leaves a configured range. Ratios can overstate small changes among rare tokens and understate large absolute changes among likely tokens. A move from 0.01 to 0.06 produces a 6x ratio despite changing by five percentage points, while a move from 0.90 to 0.20 produces a 0.22x ratio after a 70-point change. IPO measures absolute probability drift. IPO masks a token when the absolute difference between its trainer and inference probabilities exceeds epsilon: |p[trainer] - p[inference] | > ε The mask applies to probability changes in either direction and does not depend on the sign of the token's advantage, which represents whether the sampled action performed better or worse than expected. Epsilon is the only masking parameter. An epsilon of 0.3, for example, allows a difference of up to 30 percentage points. The method adapts the trust-region idea used by DPPO, where updates are constrained to keep a policy from moving too far. IPO applies that constraint to disagreement between the rollout and training systems. Prime Intellect tested IPO on agentic software-engineering reinforcement learning with GLM-4.5-Air, without replaying mixture-of-experts router decisions and with trajectories up to 64 steps off-policy. IPO completed the run with epsilon values of 0.2 and 0.3. At the new default of eps = 0.3, IPO masked about 0.25% of tokens in the reported test. An epsilon of 0.1 masked about 2%. IcePop, PPO, and CISPO remain available as explicit loss selections. MoE training reaches 3,732 tokens per second. Prime Intellect reports a 1.15x throughput lead (approximately 13.1% faster) over torchtitan in a back-to-back supervised fine-tuning benchmark for Qwen3-235B-A22B. The test used four nodes with eight NVIDIA B300 GPUs each, a 16,000-token sequence length, Fully Sharded Data Parallel training, and eight-way expert parallelism. Vendor-reported Qwen3-235B-A22B SFT results | System | Tokens/s/GPU | MFU | | prime-rl | 3,732 | 46.6% | | torchtitan | 3,253 | 40.7% | | Difference | +14.7% | +5.9 points | Model FLOPs utilization, or MFU, estimates how much of a GPU's theoretical arithmetic capacity the workload uses. Mixture-of-experts models complicate that calculation and the underlying execution because each token activates only a subset of the model's expert layers. The release attributes the throughput improvement to several MoE-specific changes: * A unified MoE runtime and checkpoint layout now serves every supported model. * Gate, up, query, key, and value projections are fused at runtime. * MoE blocks compile as complete graphs, reducing graph breaks and launch overhead. * Loss computation uses a chunked implementation based on torchtitan's approach. * A SonicMoE BF16 backend adds per-layer selection of the MoE compute implementation. GLM-5.3 receives a separate sparse-attention path. FlashMLA handles the forward pass, while cuDNN handles the backward pass. Prime Intellect reports that the forward implementation runs 1.9 to 3.6 times faster than TileLang on H200 GPUs and that the backward implementation runs 3 to 4.7 times faster. A 12-node GLM-5.3 recipe with a 16,000-token sequence length removes context parallelism and offloading. In the reported full-model SFT benchmark, throughput increased from 465 to 835 tokens per second per GPU, while MFU rose from 22.3% to 40.1%. Native coverage expands. Every architecture supported by prime-rl now uses a native implementation. Version 0.10 adds the following model families and capabilities: * DeepSeek V4 Flash: FlashMLA and cuDNN sparse-attention kernels, plus ring context parallelism. * Qwen3.8 Flash Next: n-gram embeddings sharded by vocabulary row across data-parallel ranks, indexed sparse attention, Gated DeltaNet, hyper-connection residuals, and sigmoid-gated MoE layers. * Nemotron-H Nano and Super: Mamba-2 state resets at packed-document boundaries and Ulysses context parallelism. * Multimodal RL: built-in support for Qwen-VL and Kimi K2.5. Multimodal rollouts keep image references inline through generation and sequence packing. Earlier pipelines commonly transferred decoded pixel tensors between stages, creating a separate data path for images. Evaluation moves into prime-rl. Evaluation now runs from prime-rl through uv run eval. The command supports standalone evaluation and online evaluation during training, with resumable traces and task selection shared with vf-eval. Verifiers 0.4.0 completes the project's transition to its v1 API stack. The release removes the legacy v0 implementation and standardizes command names under the vf-* prefix, so scripts that call older commands require updates. What changes for existing users. * Training configurations that rely on the default loss will use IPO with eps = 0.3. * IcePop, PPO, and CISPO must be selected explicitly when those behaviors are required. * Evaluation scripts should invoke the prime-rl evaluation command. * Verifiers integrations using the v0 stack or older CLI names must migrate to the v1 APIs and vf-* commands. Both releases are available under open-source licenses. Prime Intellect provides additional benchmark details, configuration notes, and migration information in the release post.

Prime Intellect buys Moondream to double vision AI throughput. The neocloud is folding Moondream's open vision models, Photon inference engine, and megakernel compiler into its stack to chase cheaper multimodal inference. · 1 hr ago Read 5 min * Prime Intellect acquired Moondream, bringing open vision models and the Photon inference engine in-house. * Moondream founders Vik Korrapat and Jay Allen join to lead inference and multimodal research. * Financial terms undisclosed; Moondream had under 10 staff and $4.5M raised previously. * Photon claims roughly 2x throughput over vLLM, with 34ms end-to-end inference on a B200. * Moondream 3.1, Clef, and Clef Flash are already live on Prime Inference and OpenRouter. * Deal targets underserved robotics and computer-use agent markets on open models. Prime Intellect buys Moondream to sharpen AI inference. Prime Intellect, a San Francisco AI infrastructure provider valued above $1 billion, has acquired Moondream, the small open-source lab behind a widely downloaded family of vision-language models. The deal brings Moondream's open weights, Photon inference engine, and proprietary GPU compiler into Prime Intellect's training and serving platform. The acquisition gives Prime Intellect technology for running trained models with lower latency and higher throughput. Those gains can reduce serving costs and improve responsiveness in robotics, video processing, and computer-use agents, where models must interpret a stream of images or screen states. Founders join; terms stay private. Moondream employed fewer than 10 people and had raised $4.5 million before the acquisition. Founders Vik Korrapat and Jay Allen will join Prime Intellect to lead inference and multimodal research. | Deal detail | Status | | Purchase price | Not disclosed | | Closing timeline | Not disclosed | | Founders | Joining Prime Intellect | | Existing model licenses | Unchanged, according to the companies | | Integration plan | Photon and Moondream research will join Prime Intellect's platform | Founded in January 2024, Prime Intellect announced a $130 million Series A on July 8, 2026, bringing its reported funding above $150 million. The company also reported more than $100 million in annualized revenue that month. CEO Vincent Weißer said the acquisition would support expansion into robotics and computer-use agents, markets that often require fast visual inference. Photon targets the serving bottleneck. Moondream's public profile comes from compact vision-language models that collectively exceed 5 million monthly downloads. Photon is the serving engine beneath those models, handling the work required to schedule requests, process images, and execute operations on a GPU. Photon uses model-specific scheduling, native image processing, and custom CUDA kernels. Prime Intellect reported the following results for Moondream workloads: | Hardware or comparison | Reported result | | vLLM comparison | About twice the throughput on comparable workloads | | NVIDIA B200 | 34 milliseconds of end-to-end inference | | NVIDIA H100 | More than 60 inferences per second | These figures come from the companies' announcement. Production performance will depend on model size, precision, batch size, image resolution, request patterns, and hardware configuration, so teams should reproduce the tests with their own workloads. Photon's compiler generates model-specific megakernels, which combine several GPU operations into a larger unit of work. This reduces kernel-launch overhead and intermediate memory transfers, keeping more of the GPU occupied during inference. Automating that process can shorten the optimization work required for each model, which otherwise involves extensive manual tuning by CUDA engineers. The integration is already live. Prime Intellect had integrated part of the acquired technology by the time it announced the deal. Moondream 3.1 is available through Prime Inference with Photon handling execution. Clef and Clef Flash are also available through Prime Inference and OpenRouter. Prime Intellect describes the product plan in three parts: * Moondream: Continue developing compact multimodal models under the existing brand. * Photon: Extend the runtime and kernel work to more open models and input types. * Prime Inference: Add support for video, voice, decision models, robotics, and computer-use agents. Custom runtimes squeeze more from each GPU. Neoclouds rent GPU capacity and offer managed infrastructure for training or serving AI models. Inference providers occupy a related market, giving application developers hosted access to open models without requiring them to operate GPU clusters or serving software. Falling per-token prices have increased pressure on these providers to improve throughput, latency, utilization, and reliability. Widely used runtimes such as vLLM and SGLang provide strong general-purpose baselines. A custom runtime can improve margins when it serves more requests from the same hardware, provided those gains hold across real production traffic. The strategy resembles Modular's acquisition of BentoML earlier in 2026. Both deals combine model-serving products with lower-level compiler or runtime expertise, giving the acquiring company more control over the path from an API request to GPU execution. What developers should verify. Developers considering Moondream or Photon should evaluate the acquisition through several practical checks: * Benchmark representative traffic. Test the expected image sizes, batch patterns, concurrency, precision, and latency targets instead of relying on headline throughput. * Confirm hardware coverage. Prime Intellect says the first Photon release supports Moondream, Qwen, and Gemma on NVIDIA H100 GPUs. Support for additional accelerators and deployment environments remains unspecified. * Review each license. Prime Intellect says Moondream's existing licensing remains unchanged. The compiler is described as proprietary, and the announcement does not provide broader access terms for Photon. * Check deployment options. Teams with edge or on-premises requirements should confirm whether Photon can run outside Prime Intellect's hosted platform. * Track release continuity. Open-weight users should watch model updates, repository activity, compatibility guarantees, and deprecation policies as integration proceeds. The acquisition gives developers immediate hosted access to Moondream models and a runtime designed for efficient multimodal inference. Photon's portability, reproducible performance across broader workloads, hardware support, and long-term release policy remain the key details to watch.