CONUS · 531 Basins
Active
CAMELS-US Benchmark
Periods, canonical 12-fold PUB and 18-fold HUC-02 PUR split files, and the 4 CAMELS-US forcing categories.
Hydrological Model Intercomparison Project
Standardized benchmarking for large-sample hydrology.
Shared experimental protocols, common catchments, and open daily hydrograph archives so hydrological modeling studies can be compared directly and built on cumulatively.
Image NOT created by Daniel Klotz (hopefully he will be inspired to make one).
Code
Active
GitHub Repositories
Benchmark model implementations, spatial fold definitions, and leaderboard JSONs.
github.com ↗
Archive
DOI
Zenodo DOI Archives
DOI-minted daily hydrograph predictions and per-basin diagnostic tables.
zenodo.org ↗
Cloud Store
Live
Google Cloud Storage
Public Zarr v2 daily hydrograph archives for direct cloud-native evaluation.
console.cloud.google.com ↗
Papers
In Prep
Benchmark Manuscripts
Living intercomparison manuscripts and protocol specifications across MIP suites.
Publications →Protocol & Design
About HydrologyMIP
Hydrological models are evaluated on identical catchments and dates within each benchmark suite, stratified by spatial regime and model inputs, and backed by DOI-minted prediction archives.
Experimental Design
- Common Catchments
- Shared catchment sets and target time series across all participating models in a benchmark.
- Blind Holdout Windows
- Strict separation between calibration/training periods and blind evaluation periods.
- Evaluation Metrics
- Leaderboards can be sorted by any metric. For rainfall–runoff models we use the metrics by Gauch et al. (2023).
Spatial Regimes & Inputs
- Spatial Regimes
- In-sample (gauged) locations, out-of-sample (ungauged) locations (PUB), and out-of-sample regions (PUR).
- Model Inputs
- This MIP is designed to test model structures and not end-to-end modeling systems. Different leaderboards for different model inputs help isolate differences in model structures.
- Open Archives
- Model predictions archived in a DOI-minted repository are required for each model entry.
Source Categories
- Literature
- Metrics were extracted from prior published literature where experiments used the same splits as a MIP and are therefore directly comparable.
- MIP Re-run
- Models from prior published literature that were reimplemented and rerun on the standard splits for a MIP.
- Archive
- Scored from publicly archived model predictions released with prior studies.
- Contribution
- New model runs submitted directly to this MIP by the community.
MIP Benchmark Specifications
Global · Multi-Continent
In Preparation
Caravan Benchmark
Global evaluation periods, k-fold PUB and continental/regional PUR splits, and global forcing categories.
Caravan Protocol & Splits →
CAMELS-US Benchmark Specification
Catchment screening, temporal windows, spatial cross-validation splits, and forcing categories for the CAMELS-US Daily Deterministic MIP.
Catchments & Windows
- Catchment Set
- 531 CAMELS-US basins (<2,000 km², ≤10% area error screening)
- Training Window
1999-10-01to2008-09-30- Evaluation Window
1989-10-01to1999-09-30
Spatial Splits
- Gauged Simulation
- In-sample locations: calibrated/trained on all 531 basins during
1999-10-01to2008-09-30and evaluated on1989-10-01to1999-09-30. - 12-Fold Random PUB
- Out-of-sample locations: 12 random folds (44–45 basins/fold) trained on 11 folds and evaluated on the held-out fold.
- 18-Fold HUC-02 PUR
- Out-of-sample regions: 18 USGS HUC-02 regions trained on 17 regions and evaluated on the held-out region.
4 Forcing Categories
- Single CAMELS
- Driven by a single standard CAMELS-US forcing dataset (Daymet, Maurer, or NLDAS).
- Multiple CAMELS
- Driven simultaneously by two or three standard CAMELS-US forcing datasets (Daymet, Maurer, NLDAS).
- Lagged Streamflow
- Ingests lagged observed streamflow or data assimilation alongside meteorological inputs (Gauged only).
- Freeform
- Driven by non-CAMELS or external forcing products (e.g., AORC, EM-Earth, ERA5-Land).
Caravan Benchmark Specification
Global multi-continent catchment set, temporal holdout periods, spatial cross-validation splits, and forcing categories for the Caravan MIP.
Catchments & Windows
- Catchment Set
- Standardized global Caravan benchmark basins across multiple continents and hydroclimates.
- Training & Evaluation
- Fixed calibration/training period and blind out-of-sample evaluation holdout period.
- Evaluation Metrics
- Sorted by any metric across the Gauch et al. (2023) rainfall–runoff diagnostic suite.
Spatial Splits
- Gauged Simulation
- In-sample locations evaluated on the blind temporal holdout across all Caravan benchmark gauges.
- Global k-Fold PUB
- Out-of-sample (ungauged) locations evaluated via k-fold cross-validation across global basins.
- Regional / Continental PUR
- Out-of-sample (ungauged) regions evaluated by holding out entire hydroclimatic or continental regions.
Forcing Categories
- Single Caravan Forcing
- Driven by standard Caravan ERA5-Land meteorological forcings and static catchment attributes.
- Multiple Global Forcings
- Driven by ERA5-Land combined with additional global products (e.g., MSWEP, CHIRPS, GLDAS).
- Lagged Streamflow & Freeform
- Separate leaderboards for models ingesting lagged streamflow (Gauged only) or freeform external inputs.
Benchmarks
Leaderboards
531 Basins · 1989-10-01 to 1999-09-30
10 Tables
CAMELS-US Daily Deterministic
Gauged, 12-fold PUB, and 18-fold HUC-02 PUR across single-CAMELS, multi-CAMELS, lagged streamflow, and freeform forcings.
Open Leaderboards →
Global Catchments · ERA5-Land & Multi-Forcing
In Preparation
Caravan Global Daily Streamflow
Global large-sample intercomparison across Gauged, k-fold PUB, and regional PUR regimes using standardized Caravan basins and forcings.
Contribute a Model →
CAMELS-US Daily Deterministic
Regime
Participating Models
Machine learning
Hybrid
Conceptual
Process-based
Statistical
Community Submissions
Contribute a Model
Submit a model by linking a DOI-minted prediction archive and specifying the target benchmark, spatial regime, and model inputs.
Submission Rules — CAMELS-US
-
Blind Evaluation Window
Select architectures, hyperparameters, and early stopping strictly on
1999-10-01to2008-09-30; evaluate blindly on1989-10-01to1999-09-30. - Canonical Spatial Folds Use the canonical 12-fold PUB and 18-fold HUC-02 PUR basin splits (531 basins) with zero target-basin leakage.
- Forcing Inputs Declare all meteorological and lagged streamflow inputs used; the submission system automatically places the model into the matching forcing category (Single CAMELS, Multi-CAMELS, Lagged Streamflow, or Freeform).
- DOI-Minted Archive Link a DOI-minted repository (Zenodo or HydroShare) containing daily discharge predictions; the backend resolves, inspects, and scores the archive automatically.
Submit Entry via DOI Archive
Manuscripts
Publications
In Preparation
CAMELS-US Benchmark
A CAMELS-US Model Intercomparison Project
Nearing, G., et al. (in preparation)
Forum
New Comment