Does a Prediction Market Beat the Weather Service?

Abstract. Kalshi’s temperature ladders scored against the National Weather Service’s own probabilistic guidance, the National Blend of Models, on 26,583 contracts across 6,023 city-days in eight US cities, 2021 to 2025. The market is the better forecast in every city: Brier 0.1355 against 0.1564, a clustered t of 22. It is not relaying the bulletin, and its edge is resolution, separating easy days from hard ones, which the guidance cannot do.

1. The question

Prediction market prices are read as probabilities, but they are rarely scored against a professional forecast of the same quantity, because for most events no such forecast exists. Daily maximum temperature is the exception. The National Weather Service publishes, through the National Blend of Models, an operational probabilistic forecast of exactly the quantity Kalshi’s temperature ladders settle on, at the same named stations, archived and free. So the comparison nobody usually gets to make is available in full: a retail market against the government’s own guidance, 26,583 contracts across 6,023 city-days in eight US cities, August 2021 to November 2025.

2. The answer

The market wins, in all eight cities, and not narrowly.

Two results give the finding its content. The market is not relaying the public bulletin: with both forecasts on the right-hand side, the market enters at essentially one and the guidance collapses to 0.16. And the advantage is not sharpness traded against calibration; the market is better on both terms.

3. The mechanism, which is not the obvious one

The guidance is not underdispersed. Standardising outcomes by its own stated standard deviation gives a variance of 1.002 on 4,407 events. What it cannot do is vary its spread with the day: where it states one degree of uncertainty the realised dispersion is 1.44, and where it states four degrees, 0.82. Its uncertainty is close to regime-invariant, so it fails to separate easy days from hard ones. That is a resolution deficit, not a calibration one, and it is the kind a market full of people looking out of the window can exploit.

4. So what

Two things travel beyond weather. Where a market is thick enough, its price is a professional-grade probabilistic forecast that costs nothing to read; a desk with weather in its book is leaving the better forecast on the table. And the guidance lost on resolution, not calibration. It was right on average and could not tell easy days from hard ones. That is the failure mode of any model stating near-constant uncertainty, a value-at-risk model included, and the test used here is the template for auditing any forecaster who claims to know when they do not know.

5. What this does not claim

This is not a demonstration that crowds beat meteorologists. The Blend is automated statistical guidance, not the official human forecast; the market trades after the guidance publishes and may use it freely; and the result is one settled quantity on one venue. The paper states each limit and the timing in full.