Case study: Ada builds a real-time guardrail for their agents with 50% fewer false positives. With Oumi's technology, Ada's custom guardrail model beat GPT-4.1 Mini by 4% in accuracy while slashing latency and false positives rate By Stefan Webb Table of contents Problem. Ada builds AI-powered customer service agents deployed across industries including fintech, e-commerce, SaaS, and travel. For every deployment, each action the agent takes must strictly align with the customer's operating policies - any deviation risks damaging the customer relationship. To enforce this, Ada runs a real-time adherence check: for every proposed agent action, a guardrail model evaluates the action against the customer's standard operating procedure (SOP) and classifies it as "adherent" or "non-adherent." Because this check runs live during customer conversations, latency is critical. Ada's existing guardrail relied on GPT-4.1 Mini. While reasonably accurate at 87.5%, the model introduced unacceptable latency for real-time use. Ada also wanted greater ownership and control over the model powering this critical system. Solution. Ada partnered with Oumi to develop a custom fine-tuned guardrail model optimized for policy adherence classification. Since Ada had only around 20 labeled examples initially, the team built a fully synthetic data pipeline. Using Oumi's platform, they generated 250 SOPs spanning Ada's priority domains, varying attributes like complexity and tool count to ensure diversity. From these, they synthesized thousands of agent conversations with controlled variation across customer clarity, conversation length, and adherence outcomes. This data was used to fine-tune Qwen3 4B, a compact language model well-suited to the latency requirements of Ada's real-time pipeline. Outcome. The fine-tuned 4B-parameter model outperformed GPT-4.1 Mini across all key metrics (see table above). The bottom line: By moving from a general-purpose hosted model to a purpose-built fine-tuned LLM, Ada gained better accuracy, lower latency, and full ownership of a core component in their AI agent infrastructure. What's next. With Oumi's technology, Ada were able to build a small custom AI model that solved pressing business needs. Theirs, however, is just one use case. Many other enterprises are discovering the benefits of custom AI models and the ease and economy of model development that the Oumi Platform makes possible. Why not try it out today and see for yourself? You only need come with your task prompt and the Oumi Agent takes it from there!

Case study: DMG achieves 6% higher quality and 100x lower costs for invoice validation. With Oumi's technology, DMG's custom AI beat GPT5.2 by 6% accuracy and 6% validity at 100x lower cost By Stefan Webb Table of contents Problem. Divisions Maintenance Group (DMG) coordinates facility maintenance across thousands of properties, managing a shared pool of contractors - plumbers, electricians, handymen - who submit invoices for reimbursement after completing jobs. Each invoice must be validated on two dimensions: validity (correct formatting and categorization) and appropriateness (reasonable charges given the job scope, profession, and pricing norms). At scale, manual review was labor-intensive, slow, and expensive, and DMG's existing automated approaches were underperforming - validity accuracy sat at just 72%, with no reliable baseline for appropriateness at all. Solution. DMG partnered with Oumi to fine-tune an ultra-small language model (Qwen3-0.6B) purpose-built for invoice classification. The small footprint was critical: DMG's volume demands and interest in edge deployment ruled out large proprietary models for production use. "Every job we handle is bespoke - even the same HVAC unit breaking down twice runs differently. I'm convinced our future is to have our own fine-tuned models. The results have only gotten better." - Kumar Srinivasan, Chief Product Officer The core challenge was data. DMG provided ~10k unlabeled invoice examples covering only plumbing, but the model needed to generalize across plumbing, electrical, and handyman domains. Oumi built a synthesis recipe to generate labeled training data, including entirely new examples for the missing domains. Random real invoice samples were leveraged as few-shot formatting examples within each synthesis prompt to increase sample diversity. Outcome. The small, specialized, fine-tuned 0.6B model developed with Oumi's technology delivered dramatic improvements across both tasks over the pre-fine-tuned model, improving on validity from 72% to 99%, and on appropriateness from 52% to 91%. Importantly, it also exceeded the large, general-purpose frontier GPT5.2 by 6% on both metrics (see table above). For context, Kimi K2 (a much larger model) achieved only marginally higher agreement at 95% and 87%. The bottom line: A sub-1B fine-tuned model exceeded frontier-scale performance on this task, enabling DMG to move toward fully automated invoice processing with high reliability. What's next. With Oumi's technology, DMG were able to build a small custom AI model that solved pressing business needs. Theirs, however, is just one use case. Many other enterprises are discovering the benefits of custom AI models and the ease and economy of model development that the Oumi Platform makes possible. Why not try it out today and see for yourself? You only need to come with your prompt and the Oumi Agent will take it from there!

Google's ai-generated answers often inaccurate, New York Times investigation reveals. The reliability of summaries produced by artificial intelligence has once again become a major concern. This issue is all the more crucial as it pertains to information access for a vast proportion of the global population. A comprehensive study, conducted by The New York Times in collaboration with the startup Oumi, provides essential clarification on this matter. This investigation utilized a corpus of 4,000 questions to precisely evaluate the performance of AI...