Hybrid Icechunk stores for serverless web mapping. 10 August 2026 This past winter, Cloudnativegeo saw one of the worst snowpacks in recent memory for the Western U.S., and Cloudnativegeo wanted to be able to explore how this season compared to last. NOAA provides seasonal snowfall data as NetCDF, a format commonly used for scientific workflows and analysis. Web maps shine for exploring a dataset quickly, but NetCDF files have not traditionally been compatible with web mapping, and transforming them into a web map-compatible format is a lot of work. Cloudnativegeo has been experimenting with a suite of tools and workflows that make visualizing archival formats like NetCDF in web maps significantly easier. Both the static map above and the interactive demo below render this NOAA data without the aid of a tile server, directly rendering bytes from the NetCDF analysis data. Web mapping primer. The standard path to visualizing data in a web map relies on map tiles, organized into a gridded pyramid starting with full-resolution data at its base, and reduced-resolution multiscale levels above, typically in the Web Mercator projection. As you pan and zoom, the web map requests the tiles relevant to its current extent and zoom level from a server. These tiles can be generated on-the-fly from the source data, but this comes with performance, compute, and maintenance costs. Alternatively, you can generate tiles ahead of time and store them, but this leads to data duplication and storage costs. Ideally, Cloudnativegeo would just visualize the source data directly! Web maps without a tile server. Recently, CarbonPlan released zarr-layer, an open source library for rendering gridded data directly in the browser. Given a Zarr store, zarr-layer reprojects the data and renders it in a web map without requiring an intermediary tile server. This approach works great if you have full control over your data and can store it as Zarr, but there are petabytes of gridded climate and weather data that are stored in archival file formats such as NetCDF/HDF5, GRIB, TIFF, and others. Fortunately, tools like VirtualiZarr let you read these archival data formats as if they were Zarr. These virtual Zarr references are stored in the Icechunk format and don't duplicate the data, but instead point to byte ranges inside the original files. To access these stores in a browser, Cloudnativegeo built icechunk-js, a lightweight Icechunk reader. Tying together a few tools - VirtualiZarr + icechunk-js + zarr-layer - Cloudnativegeo can render archival gridded datasets directly in the browser without a server or duplicate data. For many lower resolution datasets, this combination works as-is. Viewing broad swaths of higher resolution data, on the other hand, still requires multiscale generation for acceptable web performance. Fortunately, these multiscales are lightweight and easy to add to its virtual store. Adding multiscales to virtual stores. Direct access isn't necessarily fast access. High resolution data isn't relevant for zoomed-out views, but without multiscales, Cloudnativegeo'd be forced to load all of the data. Multiscales are pre-generated coarsened versions of data at multiple zoom levels. The Zarr community has recently converged on a standardized way of representing these coarsened versions of high resolution spatial data: the multiscale spec. To streamline the multiscale creation process Cloudnativegeo released topozarr, a library for adding multiscales and associated metadata to Zarr and Icechunk stores. Cloudnativegeo can use this to create a "hybrid" store where the full resolution data is stored as references to the original analysis data, with relatively lightweight multiscales added on. This one-time operation is cheap compared to full tileset generation because no reprojection, regridding, or duplication of the data is required. Direct access to data. Traditional web maps work with visualization-specific representations of the data, but the model Cloudnativegeo describe here streams the raw data to the browser. Since the data in the map is the actual analysis data, not an image representation, Cloudnativegeo can perform computations on it directly, revisualize it dynamically to hone in on interesting trends, or query regions of interest for exact values or averages. As a quick example, the map below shows the difference in each grid cell's snow depth between this past winter and the previous winter. It also indicates whether the current zoom is rendering NetCDF data directly or the multiscale overviews. Caveats: Chunking and bucket access. Since Cloudnativegeo is rendering the underlying analysis data directly, Cloudnativegeo is subject to the choices its creators made about chunking and codecs. Unfortunately, the chunking scheme chosen for analysis is not always best suited for web visualization. In its snowpack example, the underlying chunk scheme of the NetCDF data is in latitudinal strips from one end of the continent to the other, which is far from ideal for regional maps. To see the snow depth of your planned ski line in the Bitterroot Mountains, full strips of data stretching from Seattle to New Hampshire are loaded. For some purposes, the performance is acceptable, but other chunking schemes might make this approach untenable. Bucket access to the underlying dataset can be a problem too. If the bucket hosting the analysis data has access restrictions, or if Cross-origin Resource Sharing (CORS) is not enabled, this approach will not work. NOAA's hosting setup for seasonal snowpack data, for instance, doesn't have CORS enabled, preventing cross-origin access via browsers. As a workaround, Cloudnativegeo has mirrored the original NetCDF on S3 for this demo. Try it out. This work is exploratory, and there are many underlying data formats Cloudnativegeo haven't tested out yet. All of the pieces are open source and Cloudnativegeo'd love to see what you build. The maps above are a single self-contained HTML file, which you can read or fork at icechunk-maps. Please open issues on GitHub, reach out to Cloudnativegeo at [email protected], or find Cloudnativegeo in the CNG Slack. Thanks to the Earthmover and Development Seed teams for much of the tooling and conceptual work that underlies this workflow.

Expanding the Earthmover Data Marketplace: Sylvera, Spire, Eagle Rock and carbonplan. When Earthmover launched the Earthmover Data Marketplace in January, Earthmover described it as just the beginning. Today, Earthmover is showing that growth. Earthmover is excited to announce four new data partners joining the marketplace: Sylvera, Spire, Eagle Rock Analytics, and CarbonPlan. This second cohort adds proprietary data fueled models in the persistently challenging field of sub seasonal forecasting (Spire) and expands the marketplace in an important new direction: beyond weather and atmospheric forecasting into Earth observation, forest carbon, wildfire risk, and high-resolution regional climate projections. Together, these datasets serve the growing community of teams building at the intersection of climate risk, infrastructure resilience, and carbon accountability. The momentum is real. In the two months since the Marketplace launched, Earthmover has watched data teams across energy, insurance, agriculture, and climate risk discover that accessing production-grade ARCO data - no ingestion required, no ongoing maintenance - fundamentally changes what they can build and how fast they can build it. These four new partners deepen that value considerably. What's new. Sylvera: Biomass Atlas. Forest carbon accounting has long been hampered by the limitations of traditional field measurement: small plot sizes, poor georeferencing, and allometric models that often bias toward small trees and systematically underperform on large-canopy tropical forests. Sylvera's Biomass Atlas takes a different approach. Their dataset provides spatially and temporally explicit estimates of forest aboveground biomass density (AGBD) and canopy height at 30m resolution on an annual cadence from 2000 to the present, with pixel-level uncertainty estimates throughout. The distinguishing feature is the underlying reference data: Sylvera has pioneered the use of Multi-Scale LiDAR (MSL), a peer-reviewed method combining terrestrial laser scanning at plot scale with UAV and airborne LiDAR upscaling, covering more than 250,000 hectares of forests worldwide, representing over $10 million in data acquisition and processing. Conventional allometry often introduces significant uncertainty - ranging from 15% to 30% - into AGBD estimations. In contrast, MSL technology achieves a 3% error margin and is unbiased against destructive measurements. These high-quality MSL datasets offer the most accurate foundation available for training Biomass Atlas earth observation models. Courtesy: Sylvera The result is a dataset suitable for carbon project monitoring, jurisdictional REDD+ accounting, and AI model training at a scale and accuracy that was previously unavailable in a ready-to-use format. Key variables: * above_ground_biomass_density, canopy_height * above_ground_biomass_density_standard_error, canopy_height_standard_error * Binary masks for gap-filled and MSL-calibrated pixels A 180,000-hectare sample covering Pará, Brazil is available for immediate exploration. Full global dataset access is available on request. Spire: ai-driven sub-seasonal-to-seasonal forecasts. The 6-46 day forecast range has long been the hardest horizon to get right - too far out for numerical weather prediction to be skillful, too near-term for seasonal climate models to be useful. Spire's AI-S2S model was built specifically for this gap. Spire's sub-seasonal-to-seasonal forecast system runs daily, producing 200 ensemble members out to 46 days on a 0.5° global grid, designed and trained entirely in-house, independent of any external physical or AI-based forecast system. The model is trained and fine-tuned on ERA5 reanalysis, ECMWF IFS analyses, and Spire's proprietary data, with all inputs processed into daily statistics prior to training. The result is a fully probabilistic forecast product that gives users a rich picture of uncertainty at extended range. Courtesy: Spire The dataset is organized into four Zarr groups - ensemble mean and standard deviation (mean_stddev), percentile distributions (percentiles), climatological probability forecasts (probabilities), and anomalies relative to ERA5 1991-2020 climatology (anomalies) - covering surface and upper-air variables including temperature, wind, precipitation, pressure, radiation, geopotential height, and specific humidity across four pressure levels. Each variable is chunked at (1, 1, 361, 720) along the reference time, step, lat, and lon dimensions, meaning a single global map is always a single-chunk read. The dataset updates daily on a rolling 15-issuance window, making it well-suited for operational energy trading, agricultural planning, and insurance risk pricing workflows that depend on fresh extended-range outlooks every morning. Eagle Rock Analytics: WRF dynamically downscaled climate projections. California's energy system faces myriad decisions about grid infrastructure, building codes, and demand forecasting that require understanding climate conditions not at a global scale, but at a neighborhood and subregional scale. Eagle Rock Analytics, on behalf of Cal-Adapt, working with datasets produced at UCLA and funded by the California Energy Commission and the Amazon Sustainability and Diversity Initiative, has contributed exactly that. Their WRF dataset provides dynamically downscaled CMIP6 projections at 3 km, 9 km, and 45 km resolutions over California, WECC, and the Western United States respectively - across four climate scenarios (historical, SSP2-4.5, SSP3-7.0, SSP5-8.5), eight carefully selected GCMs, and three temporal resolutions (hourly, daily, monthly). The data is produced using the Weather Research and Forecasting (WRF) model, which preserves dynamical consistency across variables - temperature, wind, humidity, and pressure are always physically coherent with each other, unlike statistical downscaling methods where variables are processed independently. Courtesy: Eagle Rock Analytics Five of the eight GCMs include a-priori bias correction against ERA5 reanalysis, and are the recommended starting point for energy sector applications. The dataset is structured as a single Icechunk repository with hierarchically organized Zarr groups, chunked to support both spatial snapshot queries and point time series efficiently. This dataset is central to California's Fifth Climate Change Assessment and supports use cases from heating and cooling degree day analysis to WECC-wide grid reliability planning. Key variable highlighted: t2 - air temperature at 2m, covering 1980 through 2099. CarbonPlan: Open Climate Risk wildfire dataset. As wildfire risk has moved from a regional concern to a foundational input for property insurance, energy infrastructure planning, and municipal resilience, the need for reproducible, open, methodologically transparent risk datasets has grown alongside it. CarbonPlan's Open Climate Risk wildfire dataset delivers exactly that. It provides wildfire risk estimates across the contiguous United States at 30 meter resolution, with two time horizons - approximately 2011 and 2047 - enabling direct comparison of how risk is projected to shift under climate change. The dataset is built entirely on free, public input data and is fully reproducible, making it suitable for regulated use cases where auditability matters. Core variables include annual burn probability (bp_2011, bp_2047) and annual relative risk to potential structures (rps_2011, rps_2047), alongside reference variables for cross-validation. Data is provided in the WGS84 CRS at ~0.00028 degree resolution and chunked regionally for efficient access. Courtesy: CarbonPlan This is a natural complement to the climate projection data now available through Eagle Rock on the marketplace - together they give teams a path from physical climate projections to actualized risk estimates at the asset level. Why this cohort matters. Its first cohort established the marketplace as the home for production-grade weather and atmospheric forecast data. This cohort signals something broader: the Earthmover Data Marketplace is becoming the central access point for the full range of environmental datasets that climate-aware applications require - from forest carbon and wildfire risk to high-resolution regional projections and Earth observation. Every dataset in the marketplace is available in Icechunk format, meaning it integrates directly into Xarray and Zarr workflows with no custom ingestion. Subscribers receive incremental updates via Icechunk's ACID transaction model, and all data is accessible through the same consistent API regardless of the underlying provider. If you're building in energy, insurance, climate risk, or carbon markets, these datasets are ready for you now. Go ahead: Explore the marketplace. And if you have a dataset that belongs here, reach out. Earthmover is onboarding new providers continuously.