OpenAI has partnered with Broadcom to develop Jalapeño, its first custom chip designed for large language model inference. The move follows similar efforts by Google, Amazon and Meta to reduce computing costs through proprietary hardware. Inference accounts for two-thirds of total compute spending, according to Spheron. However, custom chips won't immediately lower costs for end users unless companies like OpenAI reduce their API pricing. The shift reflects broader industry challenges with AI costs. A global compute shortage is affecting companies from frontier labs to regional startups, with Southeast Asian SMEs potentially being priced out. Meanwhile, San Francisco Compute is addressing the problem by offering a marketplace where companies can sublease excess computing capacity, helping them avoid overcommitting resources to long-term contracts.

SF Compute has raised $40 million in a new funding round, valuing the company at $300 million. This investment highlights a growing interest among investors in rethinking the financing of AI infrastructure.