Confluent expands real-time AI capabilities with new security and developer tools. May 20, 2026 Confluent, an IBM company and the data streaming pioneer, today announced new capabilities in Confluent Intelligence and Confluent Cloud that streamline how real-time artificial intelligence (AI) applications are built and secured. These updates remove the security and complexity barriers that stop organizations from moving AI workloads into the real world. Confluent unifies the AI life cycle with tools that developers already live in, integrating Apache Flink pipelines with dbt (data build tool) and introducing a fully managed Model Context Protocol (MCP) server and Agent Skills that let AI manage streaming operations. With automated personally identifiable information (PII) redaction and private connectivity to external models via Azure Private Link, Confluent also embeds enterprise-grade governance directly into the data streams. "Most AI projects fail before they reach a single customer because the data layer breaks down," said Sean Falconer, head of AI at Confluent. "Teams have the models and the mandate, but security risks and fragmented data stop them from shipping. We're fixing that by making the streaming layer the foundation for secure, production-ready AI." The problem is widespread, according to a McKinsey report that says, "... eight in ten companies cite data limitations as a roadblock to scaling agentic AI." Root causes are often tied to security teams blocking data from entering AI pipelines due to exposure risks and developers losing hours to tool-switching to inspect and manage the data streams their AI depends on. The resulting slow, manual process turns what should be a fast iteration cycle into a bottleneck. Confluent Cloud and Confluent Intelligence form the data streaming foundation for production-ready AI that continuously processes historic and real-time data and delivers it as trusted context into AI applications. New capabilities add the security controls and developer tooling that high-stakes industries require. Natural language operations allow developers to use Confluent MCP as a control plane, enabling AI to build, manage, and debug streaming operations using natural language. Agent Skills add a second layer, encoding best practices and workflows so those operations are executed consistently and in line with organizational standards. Together, they enable developers to create and continuously improve real-time applications using AI-powered tools, bringing streaming into modern, agent-driven development workflows. This is generally available for Confluent Cloud. Automated data privacy introduces a new built-in ML function for PII detection and redaction that protects sensitive information directly in Flink SQL, without custom code, external services, or moving data to a warehouse first. This unlocks more AI use cases across highly regulated industries such as financial services, healthcare, and insurance. It is available in early access for Confluent Intelligence. Secure connectivity support for Azure Private Link ensures that AI workloads stay off the public internet with secure, private paths to calling external models and querying external tables. Flink jobs can securely connect to Azure-hosted services such as Azure OpenAI, Azure SQL, and Cosmos DB over Microsoft's private backbone. This capability is generally available on Confluent Cloud. Unified engineering workflows are enabled through the free open source dbt adapter that brings Flink SQL on Confluent Cloud into dbt, the industry-standard framework data engineers use to build and manage data pipelines. Teams can immediately define, test, and deploy streaming pipelines using the same dbt commands and project structure they rely on today. This lowers the barrier to Flink adoption and makes it easier to extend existing data workflows into real-time use cases. It is generally available on Confluent Cloud. Confluent also provides flexibility with additional model support, including support for TimesFM models for robust anomaly detection as well as Anthropic and Fireworks AI models, which developers can directly use in Flink stream processing workflows to build sophisticated real-time AI applications. Highlights include the general availability of the Real-Time Context Engine, which continuously delivers fresh, governed context for AI applications, and new fully managed connectors in Confluent Cloud that further simplify data integration. These capabilities extend recent announcements at IBM Think that integrate Confluent Cloud further into IBM solutions. With Confluent, watsonx.data delivers an AI-ready data foundation and a real-time context layer for AI across hybrid environments.

Confluent moves Schema IDs to Kafka headers to simplify schema governance. Write for infoq. Feed your curiosity. Help 550k+ global senior developers each month stay ahead. Get in touch Confluent has introduced a new approach to managing schema metadata in Apache Kafka by enabling schema IDs to be stored in message headers rather than in the payload. The update is designed to simplify data governance and enable teams to adopt schema validation without changing existing event formats. The feature builds on Kafka's native header support and integrates with Confluent Schema Registry, which is widely used by organizations managing event-driven architectures across microservices, analytics pipelines, and data platforms. In traditional Kafka deployments using Confluent's wire format, schema IDs are embedded directly in the message payload. This ensures consumers can correctly deserialize events, but it tightly couples schema metadata with the data itself. Over time, this coupling complicates schema evolution, especially in environments where multiple teams and systems consume the same event streams. It also increases coordination overhead when schema changes are introduced across producers and consumers. With the new approach, schema identifiers are stored in Kafka record headers while the payload remains unchanged. Consumers retrieve the schema from the schema registry at runtime using the ID in the header. This maintains compatibility with formats such as Avro, Protobuf, and JSON Schema while reducing dependence on tightly coupled wire formats. Schema resolution is decoupled from the payload, making event streams more flexible and easier to integrate across downstream systems and tooling. Schema handling before and after moving schema IDs to Kafka headers (Source: Confluent Blog Post) Patrick Neff, CSTA Team Lead CEMEA at Confluent, highlights the importance of schema governance in enabling reuse across streaming and analytics systems in a LinkedIn post. Schemas are the key enabler for unlocking the full value of your data. The header-based approach also supports incremental adoption. Organizations can introduce schema governance without large-scale rewrites or coordinated changes across all producers and consumers. Schema IDs can be attached to existing event streams, allowing teams to gradually adopt stricter schema management practices while maintaining backward compatibility. Gunnar Morling, Technologist at Confluent, emphasized improved interoperability with storage systems and downstream processing frameworks in post. Schema ids into Kafka message headers rather than the message payload is a massive quality of life improvement: payloads become valid, self-contained. Separating schema metadata from payloads enables independent evolution of producers and consumers, with validation centralized in the schema registry. This reduces coordination overhead and simplifies schema evolution at scale. It also improves interoperability with tools like Apache Flink and analytics or ML systems by enabling consistent reuse of structured event data across pipelines. David Araujo, Director of Product Management at Coflunent, describes how the feature enables zero downtime and client-independent adoption patterns. By moving schema IDs to headers, you can attach schemas to existing data in Kafka without touching payload formats. The transition may require updates to Kafka connectors and downstream tools that assume schema metadata is embedded in payloads, creating a period where both approaches may coexist, depending on ecosystem readiness. The feature is available in Confluent Cloud and is expected in Confluent Platform with Schema Registry support under existing licensing models. Leela kumili. Leela is a Lead Software Engineer at Starbucks with deep expertise in building scalable, cloud-native systems and distributed platforms. She drives architecture, delivery, and operational excellence across the Rewards Platform, leading efforts to modernize systems, improve scalability, and enhance reliability. In addition to her technical leadership, Leela serves as an AI Champion for the organization, identifying opportunities to improve developer productivity and workflows using LLM-based tools and establishing best practices for AI adoption. She is passionate about building production-ready systems, enhancing developer experience, and mentoring engineers to grow in both technical and strategic impact. Her interests include platform engineering, distributed systems, developer productivity, and bridging technical solutions with business and product goals. This content is in the Apache Kafka topic.

MongoDB, Inc. (MDB) appoints Ryan Mac Ban as Chief Revenue Officer. Jeff Lewis MongoDB, Inc. (NASDAQ:MDB) is one of the 8 Best Debt Free Stocks to Buy Right Now. On March 24, 2026, MongoDB, Inc. (NASDAQ:MDB) appointed Ryan Mac Ban as Chief Revenue Officer, effective April 27. Ryan Mac Ban brings more than 20 years of experience, most recently serving as Chief Revenue Officer at Confluent, and previously holding senior roles at UiPath, VMware, and Cisco. Outgoing CRO Paul Capombassis will support the transition through Q2. On March 23, 2026, Mizuho analyst Siti Panigrahi upgraded MongoDB, Inc. (NASDAQ:MDB) to Outperform from Neutral with a price target of $325, up from $290, citing a "compelling" setup following fiscal Q4 results. Siti Panigrahi said the company's growth profile "has inflected meaningfully," pointing to fiscal 2026 net customer additions rising 60% year over year, net revenue retention improving from 118% to 121%, and headcount growing just 1% against 23% revenue growth, which signals a shift toward "efficient, durable growth." Mizuho also highlighted AI as a "structural tailwind," noting increased application creation and database demand, and views the fiscal 2027 outlook as conservative. On March 16, 2026, Morgan Stanley lowered its price target on MongoDB, Inc. (NASDAQ:MDB) to $335 from $440 and maintained an Overweight rating, keeping revenue estimates largely unchanged while assuming more gradual margin expansion and applying a lower valuation multiple. MongoDB, Inc. (NASDAQ:MDB) provides a general-purpose database platform through cloud-based, enterprise, and community offerings. While Katzenmarshall acknowledge the potential of MDB as an investment, Katzenmarshall believe certain AI stocks offer greater upside potential and carry less downside risk. If you're looking for an extremely undervalued AI stock that also stands to benefit significantly from Trump-era tariffs and the onshoring trend, see its free report on the best short-term AI stock.