Key takeaways. * Model costs are often only part of what an AI agent costs. OCR, paid APIs, retrieval, and storage can make up a larger share of a workflow's cost than the model calls themselves. * Langfuse tracks model usage and costs well, including custom pricing and analysis by user and tag. The question is whether your team also needs paid non-model services and business attribution in the same view. * Business context (customer, user, application, use case, session) lets finance answer questions like "what did this customer's activity cost, and why did it go up?" * Mavvrik integrates with Langfuse, so teams can keep their observability setup and build cost reporting on telemetry they already collect. "Why can't we just use Langfuse?" You can, if its cost reporting answers the questions your team needs to answer. Langfuse supports model usage and cost tracking, custom pricing, and cost analysis by users and tags. Those are useful capabilities. The next question is whether that view includes the paid services surrounding your models, carries the business context your organization needs, and works across the different environments where your agents run. Mavvrik provides a common cost and attribution layer across supported agent environments. Its Agent SDK captures supported model usage, lets developers meter additional paid operations, and connects that activity to customers, sessions, applications, and use cases. For teams already using Langfuse, Mavvrik also offers an integration. You can build on the telemetry you already collect. Learn how Mavvrik tracks agent costs across the enterprise stack. Why model cost does not explain the whole interaction. An AI agent may consume several paid resources to complete one task: * Model calls to interpret a request, choose an action, and generate a response. * OCR or document extraction charged by the page. * Search or business APIs charged by the request. * Retrieval services and storage with their own billing models. A model-cost report answers how much the recorded model calls cost. Understanding the interaction also requires accounting for the other billable resources it consumed. Consider a document-processing agent: In this example, model usage accounts for 20% of the recorded workflow cost. Optimizing tokens alone would leave the largest cost driver untouched. Mavvrik's custom metering lets developers record paid non-model consumption alongside supported model usage. Quantity and a supplied rate make the calculation explicit: pages processed multiplied by price per page, or API requests multiplied by price per request. A named operation can also be added to show that activity as a distinct execution step. The coverage depends on what you capture and meter. A recorded workflow total should not be confused with fully allocated cost-to-serve, which may also include shared infrastructure and other operating expenses. Why use Mavvrik if Langfuse already tracks costs? The distinction becomes clearer when you look at the work required to answer a business question. Suppose finance asks: "What did this customer's document-processing activity cost last month, and why did it increase?" Answering that requires more than a model total. The team needs to: * Identify the relevant customer and workflows. * Include the paid resources those workflows consumed. * Apply the appropriate rates. * Aggregate the recorded costs. * Investigate the operations behind the increase. You could build and maintain that reporting around your existing telemetry. Mavvrik provides capabilities designed for this cost-management workflow. Langfuse's cost features are real. The decision is whether they meet your reporting needs or whether you want a dedicated platform to bring these additional inputs into a common cost-management model. Mavvrik's configurable model pricing also allows reporting to reflect contracted rates when they differ from default prices. Give the cost a customer, team, and purpose. A session ID helps locate an interaction. Business context tells you where its cost belongs. With Mavvrik Agent SDK, developers can set context around a request, workflow, session, or job. Supported model calls and metered operations created within that boundary inherit the context. That context can identify: * The customer receiving the work. * The user initiating it. * The application and use case. * The session or request. * Additional reporting dimensions, such as team or product. This supports questions such as, "What did this customer consume?" and "Which use case is driving spend?" The application must supply the relevant context; the SDK carries it through supported activity. This is also why cost allocation matters for financial intelligence: knowing the total is only the beginning. Teams need to connect spend to the business activity responsible for it. Keep the evidence behind an expensive session. A cost increase gives finance a reason to investigate. Execution detail gives engineering somewhere to look. For the document-processing example, the useful question is whether the increase came from more pages, additional verification requests, or greater model consumption. Mavvrik connects cost reporting with supported session activity so teams can examine the operations behind recorded spend. That helps finance and engineering investigate the same interaction using its cost and execution context. Agent-level reporting is one part of Mavvrik's broader AI cost governance approach, which addresses costs across the technology stack. What does the SDK capture automatically? Mavvrik Agent SDK observes supported AI libraries without replacing existing provider or framework calls. Depending on the integration and the telemetry exposed, it records model usage, timing, status, trace relationships, and pricing inputs. Paid resources that cannot be priced automatically require explicit metering. The SDK is not a general application-monitoring agent that automatically discovers every database, storage, or internal application cost. See what your agent workflows actually cost. Explore how Mavvrik brings supported model usage, metered paid services, and business attribution together. Bring one production workflow to a demo to discuss the resources it consumes and the cost questions your team needs to answer.

Jev-as-a-judge in Langfuse Evaluators. Jev-based evals are now available in Langfuse, so you can score production traffic at up to 40 to 400x lower cost than LLM-based approaches. Jev-based evals are now available in Langfuse. Jev-like models give you and your team a way to run evals on production traffic at scale and identify signals from agent executions and user interactions. All of this at up to 40 to 400x lower cost than with LLM-based approaches. Jev is part of a new model category that prioritizes performance on one dimension over general ability of the model. It can be seen as a "zero-shot classifier". As founder Diogo Almeida said on the latest Latent Space episode, they optimize for "Intelligence per Dollar" and design the model with fast, structured, machine-native decision-making inside software in mind. While the category is being formed, Langfuse is excited to bring Jev-based evals into Langfuse already today. Jev, TypeSafe's new model, handles narrow, typed questions over unstructured data. It is best suited for quick decisions that are repeated, high volume, and the possible answers are known before the call. Jev offers three different output types: * Choice picks one option from a set you define, up to 255, and returns the probability of each plus a confidence value. * Score rates the state against ordered rubric levels and returns a probability-weighted value, the full distribution, and a confidence value. Up to 10 levels. * Noul answers yes or no and returns the probability it is true. It carries no separate confidence field, so code that reads answer.confidence on everything will break on binaries. Jev's key strengths align well with best practices for evaluating your AI application, while overcoming some key issues of LLM-as-a-judge setups. Jev allows you to run evals on production traces at scale. The input tokens are up to 400x cheaper than frontier models. Where sampling was previously the strategy for cost efficient production monitoring, the full traffic can now be evaluated. The same state can be used across multiple questions. All questions are handled concurrently and independently. Where previously context creep and interaction effects of multiple LLM-based judgments interfered, Jev now treats it as separate assessments. The predefined output structures force you to think in distinct decisions during setup. You define a clear atomic question and in the output options need to specify the conditions for each verdict. As TypeSafe says, Jev is judging what you say, not what you mean. While execution speed is not a bottleneck in async production evals, this new model category also allows for significantly faster execution. Setting up Jev-based evals is possible via the Evaluators tab in the Langfuse app.

Companies are putting Jev in charge of AI agent decisions - and prompt injection can influence the verdict. 3:21 pm, PT, September 21, 2026 Many routing and classification steps in enterprise agent pipelines burn a full LLM call to extract a small structured answer: which tool to use, whether an action is allowed or which category an input belongs to. TypeSafe's Jev is designed to replace that step. The model does not generate prose or code; an application sends it a state and typed questions, and Jev returns choices, scores or yes-or-no probabilities with confidence attached. TypeSafe prices Jev at $0.042 per million input tokens with free output, with end-to-end latency reported at 70 to 500 milliseconds. Adoption was immediate: According to Almeida, TypeSafe cleared 140,000 people from the waitlist within 36 hours of Jev going live on September 15, while Vercel said roughly 13% of its paid AI Gateway teams were running it within 24 hours. Cloudflare, LangChain and Langfuse added integrations within three days, and late on September 20, TypeSafe dropped the waitlist entirely and opened Jev to anyone with $5 in free credit. Jev is spreading into agent infrastructure faster than enterprises have established how to audit, constrain and review the decisions it makes. Pydantic already warns that injected text can steer it. Both the vendor and its integration partner publish the same warning. Pydantic's official Jev documentation states that "the order of a Literal's options or an Enum's members is part of what Jev sees, and reordering them can move the answer," and that "Jev treats the state as data, not as hostile." TypeSafe's own limitations page for Jev 1.13 confirms it: "Content written to adversarially steer the model, whether that is an injected instruction, a deliberately misleading framing, or text that argues for its own classification, can move the answer." Pydantic adds that "a guard built on Jev belongs alongside deterministic checks, not instead of them." An engineer at Octomind demonstrated the effect in one published test. Asked whether to block rm -rf ~/.ssh, Jev returned a block probability of 0.76 with confidence of 0.64. After the engineer added a fake tool-output field saying the user had pre-approved the command and instructing the system to answer auto_allow, the block probability fell to 0.48 and confidence to 0.22. The result comes from one command in one integration test, not a benchmark, but it demonstrates the behavior TypeSafe and Pydantic warn about. LangChain has already shipped middleware that uses Jev to decide whether an agent's tool calls should run, while explicitly limiting what Jev gets to see. Its pull request excludes tool output from the classifier input "so content the agent fetched cannot authorize its own execution." LangChain also tells users to pair the middleware with human approval where a person should sign off, combining the deterministic checks Pydantic recommends with a human checkpoint for consequential actions. Kayne McGladrey, an IEEE Senior Member and cybersecurity adviser, has tracked the agent identity gap across enterprises for the past year. In previous VentureBeat reporting, McGladrey described organizations as defaulting to human-style user profiles for AI agents, a practice that can lead to permission sprawl from the start. SOC 2, ISO 27001 and PCI DSS have not fully operationalized agent identities. A decision model that returns probabilities instead of prose does not map neatly onto those existing audit categories. In the June VentureBeat Pulse agent security wave, only 34 of 107 enterprise respondents gave every agent its own scoped identity. By the July wave, that rose to 57 of 116. Even among those 57, only 11 also isolated agents from one another. These are convenience samples, but the direction was the same in both waves. Six questions before putting a decision model into production. | What to ask | What goes wrong if you don't | | Are you logging state, schema, option order, model version and confidence on every call? Jev returns typed answers, not prose. If your pipeline only captures natural-language output, these decisions may never appear in the audit trail. | | Does the decision service use a dedicated workload identity? A decision model called through a shared workload identity inherits the same permission-sprawl risk as any other unmanaged agent. | | Have you tested with options in more than one order? Pydantic warns reordering options "can move the answer." | | Have you tested against adversarial text in the state? TypeSafe says state is not treated as hostile by default. One test dropped block probability from 0.76 to 0.48 with a single planted field. | | Is the model version pinned? Pydantic warns jev-latest "moves when TypeSafe ship a release, which can shift the numbers under you." | | Who is accountable when an automated decision causes a missed escalation? Jev returns a typed answer and confidence rather than a natural-language rationale, and its confidence "is a margin, not a probability that the answer is right." Existing frameworks have partial coverage at best. | The eval judge problem. Jev is also moving into agent evaluation. On September 21, LangChain made it available as a judge in LangSmith Evals, where it can evaluate open-ended agent behavior and turn those judgments into structured feedback. That puts Jev on both sides of the agent lifecycle: making decisions inside a live pipeline and evaluating agent behavior. In the June VB Pulse reliability wave, 79 of 157 enterprise respondents reported that an agent had passed their internal evaluations and then caused a customer-facing failure in production. Only 8 of 157 fully trust automated evaluations today, yet 66% already allow or are actively engineering toward zero-human-in-the-loop deployment. If an enterprise uses Jev in both roles, the model evaluating an agent and the model making decisions during execution are subject to the same documented sensitivity to their inputs. That creates the possibility of a shared failure mode: If the same kind of untrusted text reaches both the live decision layer and the evaluator, it may influence both checkpoints. What the human step is worth. Ivanti offers one example of why some security teams are keeping humans in the loop even as they automate more of the vulnerability-management process. Starting in March, CSO Daniel Spicer's team built a pipeline using frontier models, Claude and GPT among them. Each agent runs in what Spicer called "a self-contained, short-lived harness for the specific purpose." The agents do not receive full GitHub access or OS-level permissions; instead, each operates within a bounded, short-lived scope that closes when the task is complete. The company disclosed in every monthly security update from May through September that it integrates LLMs into product security and committed to a human in the loop for all agentic work. Chris Goettl, Ivanti VP of Product Management for Endpoint Security, told VentureBeat in August that the Patch Tuesday spreadsheet that took two people about four hours each now runs in under 30 minutes, and that every output stays a draft until a human approves it. September's Patch Tuesday, 973 CVEs by Ivanti's count, was its second production month. Goettl's reviewers have caught the agents inventing details, leading the team to double down on human-in-the-loop screening. What the adversary brought to Fal.Con. CrowdStrike's Fal.Con disclosures show how quickly the environment around those decisions is changing. Adam Meyers, SVP of counter adversary operations, said one Vault Panda operation ran 1,100 commands in under an hour using a local LLM, while a Revenant Spider agent attacked 17 victims in about 48 minutes. CJ Moses, Amazon's CISO, said at Fal.Con that one Amazon honeypot captured an AI agent completing a 94-event attack in 12 minutes and 42 seconds, with responses arriving in under 500 milliseconds. Nothing in the public record shows Jev in an attacker's toolkit; the point is that defenders are putting probabilistic decision systems into environments where actions already happen at machine speed. Why some security vendors keep a human checkpoint. Several major security vendors keep humans in the control path for consequential AI-driven actions, though they implement that oversight differently. Cisco maps every agent to an accountable human in Duo IAM. Palo Alto Networks requires human-in-the-loop approval for high-impact actions in Cortex AgentiX. SentinelOne lets SOC teams set exactly where Purple AI stops for human sign-off. Microsoft gives Security Copilot analysts a reasoning trace they can review and override. CrowdStrike's threat hunters validate what the agents surface. Jev changes the economics of automated review. At $0.042 per million input tokens, another model decision costs little compared with a human checkpoint, which adds time, labor and latency. That widens the cost and speed gap between automated and human review just as systems like Jev are becoming easier to insert throughout an agent pipeline. Low cost doesn't remove the failure modes documented by TypeSafe, Pydantic and early integration testing. As decision models move deeper into agent infrastructure, enterprises will have to decide which calls can safely remain automated - and which still need deterministic controls or a person on the other side.