Introducing Feldera scale-out: Get to real time in half the time. Ben Pfaff Chief Engineer / Co-Founder | October 20, 2026 Every data pipeline has to deal with historical data, before it can tell you anything interesting. How that happens depends on the system. In Spark, a batch job rereads your historical data on every run, recomputing its results from scratch, even when only a small fraction of the data has changed. The cost of each run and time grow with the size of your data, not the size of the incoming changes. Feldera takes a different approach. As an incremental view maintenance engine, it reads your history once, up front, then computes only what changes from then on. That one-time step is called a backfill. What is a backfill. In Feldera, a backfill is when a Feldera pipeline ingests a snapshot of your existing data and builds the internal state it needs to perform incremental updates. Feldera processes inserts, updates, and deletes as changes, and a backfill is one very large batch of changes. Once it completes, Feldera maintains every view by looking only at new changes, without ever recomputing old data again. How often will I need to backfill? * Once, when a pipeline starts fresh. A backfill happens when you run a new pipeline, or restart one with its storage cleared. * Sometimes, when your queries change. Adding views that build on existing tables and views can reuse state Feldera already has. Adding a new table or column from a source requires recomputing that table and anything downstream of it. * Got late-arriving data? No problem. A late event is just another change. It propagates through the pipeline and updates the affected views, with no reprocessing window. Feldera can ingest millions of rows per second on a single host. In most cases, this is more than enough. That said, when Feldera see 10s of TBs and/or highly complex SQL, sometimes additional horsepower is required to meet backfill SLAs, and that's precisely what Feldera's scale-out feature is for. Introducing Feldera scale-out. To improve backfill performance on the largest use cases, Feldera is introducing scale-out as a new Feldera enterprise feature. This adds the ability for a Feldera pipeline to run in a cluster that spans multiple machines. In recent POCs, Feldera has seen backfill times cut in half. Scale-out builds on Feldera's Transactions feature, which groups historical data into a single efficient operation, and then scales that work out to as many hosts as you need. What scale-out means for your team. Every system has to backfill. Scale-out makes that step faster * Get incremental pipelines live faster. Cutting backfill time means new pipelines compute real-time results sooner. * Less downtime due to a backfill. Evolve your pipelines freely, add a column or reload a source with ease, so you can meet tighter SLAs on availability. * Handle enterprise scale with confidence. Terabytes of history and billions of rows are normal for enterprise data, and scale-out grows with the workload as you add hosts. * Turn it on with one setting. Set the number of hosts, and Feldera does the rest, with no changes to your existing SQL. How it works. A Feldera pipeline divides input data into shards, one per worker thread. Each worker independently processes the data in its shard. At some points in the data analysis process, the data needs to be split up differently among workers, or "resharded". In a single-host pipeline, resharding just moves data around in memory among the workers, but in a multiple-host pipeline, much of the data needs to be sent to other hosts. Feldera does that over the network: each host maintains a connection to each of the other hosts and sends data to be resharded across it. A single-host instance of Feldera handles all of the input and output connectors on that host. For multi-host pipelines, Feldera instead evenly distributes the input and output connectors across the hosts. For input connectors, this spreads the work of communicating with the sources of the input across the hosts, making better use of CPU cycles and network bandwidth for fetching large amounts of data. For output connectors, it does the same thing for sending the output data to the output sinks. While data processing is distributed across the cluster, some tasks need to be centralized. A scaled-out Feldera pipeline has an additional process called a "coordinator" that runs in the same pod as the first pipeline. The coordinator initializes the cluster by helping each of the pods find the other ones and divvying up the input and output connectors among the pods. After initialization, its biggest job is to ensure that the pods process data together: when data arrives for an input connector for one pod, the pod starts processing it and notifies the coordinator, which tells the other pods to start processing, too, even if they have not yet received data themselves. This is important because, due to resharding, all the pods must run data processing in lockstep. The coordinator supports the Feldera pipeline API, the same one that each pipeline pod implements itself. It implements most of the API by calling into each of the pipelines. For example, when the pipeline manager requests pipeline statistics with the /stats API, the coordinator calls /stats on each of the pipeline hosts and combines their results, e.g. by summing the amount of memory used by each pipeline pod to report the total amount of memory used by the pipeline. Some of the API calls require a little more sophistication to implement across the cluster. Usually this amounts to making sure that the pipeline pods are operating in synchronization. Checkpoints, for example, require the coordinator to tell each of the pipeline pods to prepare for a checkpoint, then in some cases to process further data in lockstep until all of the input connectors across the pods are in a safe spot for a checkpoint, then to complete the checkpoint. Only if all of the pods successfully checkpoint does the overall checkpoint succeed. Transactions are another example: the coordinator ensures that all of the pipeline pods start and complete transactions at the same time rather than independently. Get started. Scale-out is available in Feldera Enterprise. For users, scaling out a pipeline only requires setting the "hosts" configuration parameter to a value greater than 1, and perhaps increasing the number of "workers" to match, since Feldera spreads the workers across the hosts. Then, when Feldera launches the pipeline, it creates the specified requested number of Kubernetes pods, which work together to execute the pipeline. A scaled-out pipeline typically has more CPU cores available to it than a single-host pipeline, since it can use the cores on multiple systems; at the same time, those systems can have fewer CPUs than a single scaled-up pod would require. Similarly, a scaled-out pipeline typically has access to more RAM, more local storage, and more I/O bandwidth than a single machine. The latter can be particularly helpful to fetch historical data more quickly. Join the Feldera community. Talk to its experts and ask them anything

Feldera is rebuilding SQL around incremental compute. Feldera is a Palo Alto data-infrastructure company building an Incremental View Maintenance engine for complex SQL. Led by CEO and co-founder Lalith Suresh, the company turns database updates into targeted computation, so teams can refresh derived data without rerunning the entire underlying workload. That sounds like a technical optimization until the cloud bill arrives. Modern data teams often recompute enormous tables because a tiny fraction of the source data changed. The ritual is familiar: schedule the batch job, provision the cluster, wait for freshness, and pay for work the system already did yesterday. Feldera is attacking that ritual at the engine level. Its DBSP-based system maintains SQL results as data changes, including complex joins, recursive queries, and sliding windows. The company says customers have moved from hours or days of latency to seconds while cutting warehouse-compute costs by 95% or more. Those are company-reported results, but they clarify why investors just funded the category rather than another dashboard on top of it. About Feldera. Feldera was founded in 2023 by five systems researchers and engineers who had been working on incremental computation problems since 2018. The company's official history traces the work through VMware, where precursor technology ran in production beginning in 2021, and into DBSP, the research foundation behind Feldera's engine. The founding team is unusually technical even by database-startup standards. Lalith Suresh is CEO; Leonid Ryzhyk is CTO; Mihai Budiu is Chief Scientist; Ben Pfaff is Chief Engineer; and Gerd Zellweger is Head of Engineering. Their backgrounds span VMware Research, Microsoft, Nicira, Open vSwitch, operating systems, compilers, distributed systems, and formal methods. This is not a company pretending that a familiar workflow becomes new when somebody adds an AI button. Feldera is commercializing a different computational model: keep the result current by processing what changed, instead of treating every update like the database has developed amnesia. The problem Feldera is solving. Batch recomputation has survived because it is easy to understand and difficult to replace. A warehouse query runs over the available data, produces a result, and runs again later. The problem is that the work often scales with the total dataset even when the useful change is tiny. Feldera's product is designed to make compute scale with the change. Teams write SQL, connect batch or streaming sources, and maintain derived views continuously. The engine supports workloads such as real-time data pipelines, AI knowledge graphs, fraud detection, event analytics, user segmentation, and fine-grained authorization. The commercial argument is not merely faster queries. Fresh data becomes practical for applications that could not justify the cost or operational burden of constant full recomputation. That matters for AI systems because agents built on stale context can make perfectly efficient decisions about yesterday. From database research to production infrastructure. The technical center of Feldera is DBSP, a general framework for incremental computation that received the VLDB 2023 Best Paper award. Feldera says DBSP allows the engine to incrementalize arbitrary SQL, including hundreds of joins, recursive logic, and large state that may exceed memory. Customer evidence is beginning to show where that theory lands. Feldera's Auth0 case study describes maintaining more than 7 billion permission checks for Auth0 FGA's Permissions Index. The company's product page also names or features teams including Oso, Hopsworks, Eviny, Procore, Nubank, Zeta Global, and Solana Vibe Station. Feldera has also published examples of customers migrating hundreds of thousands of lines of SQL from large Spark clusters to one or two Feldera nodes. The exact economics will vary by workload, and company case studies are not universal benchmarks. Still, they expose the wedge: recurring computation is expensive precisely because the same work is repeated so reliably that teams stop questioning it. Why Feldera matters right now. Enterprise AI is increasing the premium on current, structured context. Knowledge graphs, authorization systems, fraud models, and operational agents need data that reflects what changed now, not what a nightly pipeline eventually remembers. At the same time, AI spending makes infrastructure waste harder to ignore. Feldera's timing sits at that intersection. The company is not selling a new data destination. It is selling a way to keep the existing analytical and operational model current with less recomputation. If that promise holds across real production SQL, incremental view maintenance can move from a specialist database technique into a standard layer of the modern data stack. The latest financing gives Feldera more room to prove it. The company announced $21.5M across its Seed and Series A rounds, with the $15.4M Series A led by Inovia Capital and participation from Costanoa Ventures and Battery Ventures. Feldera says the capital will deepen the core engine, scalability, lakehouse integrations, and a fully managed experience. Leadership, culture, and the hiring signal. Feldera's careers page describes a global, remote-first team with high ownership, asynchronous collaboration, and direct access to the founders. It does not currently expose a list of named openings, but it invites candidates to introduce themselves and says the company is hiring across the board. That hiring signal matters because the work ahead is not only research. Feldera must turn a sophisticated engine into reliable enterprise infrastructure, make deployment and integration routine, and explain a model that asks buyers to reconsider decades of batch habits. The current team already spans core engineering, forward-deployed work, sales, operations, and product marketing, a sign that the company is moving from technical proof toward repeatable adoption. What Feldera signals for data infrastructure. Database infrastructure has a habit of making old ideas feel inevitable until the economics change. Batch recomputation became normal because compute was available and freshness could wait. AI systems are now testing both assumptions at once. Feldera's bet is that the industry will stop paying to rediscover the same answer. The company still has to prove broad reliability, simple operations, and durable economics across enterprise workloads. But the strategic question is already sharp: if only a fraction of the data changed, why should the entire bill start over? That question gives Feldera a credible category position. It is building the machinery that makes fresh data cheaper, complex SQL continuous, and real-time context less theatrical. For data leaders, builders, and infrastructure investors, that is a company worth watching because it turns one of the cloud's oldest waste patterns into a measurable engineering target.