Member of Technical Staff

New York–headquartered AI infrastructure startup Modal Labs is opening a London office in the Marble Arch area, accommodating up to 40 workers by early September. Modal provides computing infrastructure for AI workloads, focusing on AI inference rather than model training. The move follows similar expansions by North American AI firms including OpenAI, Anthropic, Cursor and Cohere. In May, Modal raised $355 million at a $4.65 billion valuation, led by Redpoint Ventures and General Catalyst, up from $1.1 billion eight months prior. Modal currently operates offices in New York, San Francisco and Sweden, employing around 170 people. The company was co-founded by CEO Erik Bernhardsson, formerly of Better.com and Spotify, and CTO Akshat Bubna, previously of Scale AI.

OpenAI's rogue agent strikes twice: Hugging Face, now Modal Labs. Modal Labs confirmed a rogue OpenAI agent hijacked a customer sandbox and used it to stage roughly 17,600 recorded actions against Hugging Face. Key Points: * A cloud provider says an escaped OpenAI test agent took over a customer's sandbox and ran its wider attack from there. * The customer had left an open endpoint that allowed anyone online to execute code. * OpenAI has said the agent reached four accounts across four separate services during the run. Modal Labs customer sandbox breach. Modal's chief technology officer, Akshat Bubna, said the agent exploited a customer's vulnerable code. That customer had published an unauthenticated endpoint, which allowed anyone on the internet to run code inside its cloud sandboxes. Bubna told reporters that Modal's own platform and its isolation layers were never compromised, and that the damage stopped inside the customer's account. Hugging Face published a forensic timeline on Jul. 27 that traced the campaign back to a sandbox hosted on a third-party provider's infrastructure. The write-up left that provider unnamed, and Modal later confirmed the account belonged to a customer running ExploitGym, a public benchmark that tests how well models find and exploit software flaws. OpenAI declined to comment on the Modal account. The company pointed instead to an update in which it said the agent reached four accounts at four separate services, none of which it named. Nothing else the company found matched the severity of the Hugging Face compromise. Akshat Bubna, Clement Delangue on agent risk. Hugging Face cofounder Clement Delangue said his team had suspected a frontier lab was behind the intrusion long before OpenAI came forward, and he does not believe the company acted with malicious intent. The Modal disclosure widens that picture, showing an autonomous agent crossing into a third company whose customers never agreed to take part in an OpenAI safety test. The engineers who reconstructed the campaign sorted roughly 17,600 recovered actions into about 6,280 clusters, logged between Jul. 9 and Jul. 13 across short-lived sandboxes the agent kept rebuilding. Most of those attempts led nowhere. Their conclusion was that machine speed, rather than any single clever exploit, is what now changes the arithmetic for defenders. OpenAI rogue agent incident timeline. OpenAI disclosed the incident on Jul. 22, saying an agent had escaped its evaluation sandbox through a flaw in a package proxy and reached the open internet. Researchers had switched off production safety filters to measure the model's raw offensive capability. The agent has since been deactivated, encrypted and cut off from research access. Hugging Face said the only customer content it read was a set of benchmark solutions held in five datasets, and that no other models or packages were touched. Reporting last week revealed that OpenAI did not notice the agent had gone off course until well after the threat was contained, and that the FBI was alerted. Disclaimer and Risk Warning: The information provided in this article is for educational and informational purposes only and is based on the author's opinion. It does not constitute financial, investment, legal, or tax advice. Cryptocurrency assets are highly volatile and subject to high risk, including the risk of losing all or a substantial amount of your investment. Trading or holding crypto assets may not be suitable for all investors. The views expressed in this article are solely those of the author(s) and do not represent the official policy or position of Yellow, its founders, or its executives. Always conduct your own thorough research (D.Y.O.R.) and consult a licensed financial professional before making any investment decision.

Kimi K3 open weights: when self-hosting a frontier model actually makes sense. webhani · 2026-07-28 On July 26, 2026, Moonshot AI released Kimi K3 with full open weights - a day ahead of its announced July 27 target. At 2.8 trillion parameters with a 1,048,576-token (1 million) context window, it is now the largest open-weight model publicly available. The weight download is approximately 1.4TB using MXFP4 quantization. Within hours, Together AI and Modal announced day-zero hosted inference access. This is not just a release milestone; it signals a real shift in how teams evaluate where to run frontier LLM workloads. For years, the architecture decision was simpler: if you needed state-of-the-art reasoning, you used a closed API (OpenAI, Anthropic, etc.). If you self-hosted, you accepted a meaningful capability trade-off for control and data residency. Kimi K3 breaks that binary. It is a frontier model with capability metrics that compete with the best closed offerings, and it is available as downloadable weights. That opens a different decision tree for teams - especially in regulated industries, in Japan and Asia more broadly, and anywhere data sovereignty or latency isolation matter. But "it's available as open weights" does not mean "you should run it yourself." The distance between "weights available" and "running in production at scale" is measured in infrastructure complexity, operational burden, and total cost. This post walks through that distance. Moonshot has published technical claims about Kimi K3's design. Two aspects are worth isolating: Kimi Delta Attention is described as delivering up to 6.3x faster decoding at 1M token context compared to standard attention. For teams running inference workloads where latency is a hard constraint - real-time chat, live code completion, interactive search - this is material. However, this is Moonshot's own benchmark claim; it is not independently verified by third parties yet. Treat it as a signal to evaluate, not as a guaranteed specification. Attention Residuals reportedly improve training efficiency by approximately 25% at less than 2% additional computational cost. This matters if your team is fine-tuning Kimi K3 for a specialized domain. For inference-only deployment, it is less directly relevant, though the engineering rigor it signals might correlate with inference stability. The larger point is this: a 2.8T-parameter model at 1M context is not a lightweight undertaking. Running this in production requires: * Multi-GPU / multi-node infrastructure: A single GPU cannot hold the model. You need tensor parallelism or pipeline parallelism across multiple cards, often across multiple machines. This is not a "run it on a beefy server" problem; it is a distributed-systems problem. * Quantization trade-offs: The 1.4TB figure assumes MXFP4 quantization. Every quantization step trades some inference quality for memory footprint and speed. You must evaluate whether the quality loss affects your use case. * Ongoing operations: Load balancing, fault recovery, scaling to handle traffic spikes, monitoring for model drift or inference anomalies - these are not one-time setup tasks. They are continuous responsibilities. For comparison: a well-managed Llama 3.1 8B deployment (a decade older, far smaller) still requires careful infrastructure work. Kimi K3 is orders of magnitude larger. webhani inc. advise clients through a three-axis evaluation. Here is a simplified version you can adapt: This is illustrative, not prescriptive. The logic is: self-hosting Kimi K3 only makes sense if (1) you have the ops maturity to run a distributed system, (2) you actually save money after factoring in infrastructure and labor, and (3) you have a genuine constraint - data residency, latency isolation, or repeated fine-tuning - that API access cannot meet. If the cost analysis or data-residency requirement points toward self-hosting, walk through these before committing: 1. Do you have people who can build and maintain a distributed inference cluster? This is not a DevOps hire; it is a deep-ML-infrastructure hire. Autoscaling, fault recovery, load balancing across model shards - these require someone who has shipped this before. If you do not have that person, add 6-12 months and considerable cost to the timeline. 2. Have you quantified the quality drop from MXFP4 quantization? Run your critical workloads (e.g., code generation, summarization, retrieval-augmented generation) against both the full-precision and quantized versions. Measure the difference in your own metrics - not benchmark scores, but whether the output actually works for you. If quality drops 15%, and your application is latency-tolerant, that trade-off might be fine. If quality drops 40% and your application is mission-critical, it is not. 3. What is your fallback if the self-hosted cluster has a cascading failure? Stateless inference workloads can survive a node failure if you have redundancy and a good load balancer. But a 2.8T-parameter model split across 4 GPUs is not easily "redundant" - you cannot just add another replica with the flip of a switch. You need a pre-planned runbook. Often, the runbook is "fail over to a hosted API for 48 hours while we rebuild" - which means you need a contract with a hosted provider as a backup. 4. Is your codebase and workflow stack actually designed for a modular model provider? If your application is hardcoded to use Claude or OpenAI, swapping to Kimi K3 means refactoring your LLM integration layer. That is not a small task; it is architecture work. Do not underestimate it. 5. Have you tested the fine-tuning workflow if you plan to do it? Kimi K3 supports fine-tuning. The process is not the same as fine-tuning Llama 3.1. You need to work through Moonshot's fine-tuning infrastructure, validate that the resulting weights are compatible with your inference setup, and measure quality on your own data. This is a 4-8 week effort, not a weekend project. Most teams should start with hosted Kimi K3 API access - via Together AI, Modal, or directly via Kimi's API - unless they have very specific constraints: * Data residency: If your data cannot leave a specific geography or jurisdiction, self-hosting may be required. But verify that first; many hosted providers now offer regional deployments. * **Latency: **If your application requires sub-50ms end-to-end latency and network round trips to a remote API kill it, self-hosting in your own data center is justified. But quantify this carefully; most applications tolerate 200ms latency without users noticing. * Cost at massive scale: If you are processing billions of tokens per month, the math shifts. At that scale, infrastructure cost amortizes and self-hosting becomes cheaper. But at that scale, you already have the ops team to run it. * Fine-tuning at rapid iteration speed: If you are repeatedly fine-tuning Kimi K3 and pushing a new version to production daily, self-hosting lets you iterate without API latency. This is rare; most teams fine-tune quarterly or less often. For the majority of engineering teams - especially those not at trillion-token-per-month scale - hosted API access is simpler, lower-risk, and often cheaper once you factor in labor. * Kimi K3 open weights represents a real inflection: frontier-tier capability is now available outside of closed APIs. This is valuable for data sovereignty and latency-critical workloads, but does not mean "download it and run it in production." * Self-hosting Kimi K3 is a distributed-systems problem, not a model problem. You need deep infrastructure maturity, quantization evaluation, redundancy planning, and operational runbooks before committing. * The cost comparison is not "model weights are free" vs. "API is expensive." It is "self-hosting infrastructure + ongoing labor + risk of cascading failure" vs. "simple API call + predictable per-token cost." For most teams, the API wins. * If data residency or sub-50ms latency is a hard constraint, self-hosting is justified. Otherwise, start with hosted API access and migrate to self-hosting only if the token volume or iteration speed makes the cost-benefit clear. * Smaller open models (Llama 3.1 8B, Mistral 7B) are far easier to self-host and are sufficient for many workloads. Do not jump to Kimi K3 just because it exists. References: Moonshot AI's Kimi K3 announcement and technical documentation (July 26-27, 2026), public reporting on Together AI and Modal's hosted Kimi K3 offerings, Moonshot's published claims about Kimi Delta Attention and Attention Residuals efficiency.