Crowd forecasts outperform the experts, a 2004 study found
In 2004, economists Justin Wolfers and Eric Zitzewitz examined a forecasting puzzle: why could prices formed through transactions among strangers rival the predictions of trained specialists? Their paper, published in the Journal of Economic Perspectives, reviewed evidence from political elections, financial markets and other uncertain events.
A familiar illustration is the price attached to a bet on a sporting outcome, with such markets listed by betting sites. Although bookmaker margins complicate the interpretation of these prices, the example introduces the question at the centre of the research: how can the expectations of numerous participants produce forecasts that outperform individual judgements?
What the 2004 study actually found
Wolfers and Zitzewitz examined existing prediction markets rather than conducting a controlled experiment. Their research assessed whether market prices provided useful estimates of future outcomes and how those estimates compared with established forecasting methods.
The Iowa Electronic Markets offered particularly revealing evidence. Since 1988, these markets had allowed participants to trade contracts linked to American presidential election results, producing continuously updated estimates of expected vote shares.
During the week preceding elections, the average absolute forecasting error was approximately 1.5 percentage points for market predictions, against 2.1 points for the final Gallup polls included in the comparison.
These figures measured deviations from actual two-party vote shares, not the success rate of predictions about election winners. The narrower distinction prevents an exaggerated interpretation of the results.
The researchers found that market forecasts were generally accurate and frequently surpassed moderately sophisticated alternatives. Their evidence did not establish an unconditional advantage over professional forecasters.
How market prices absorb dispersed information
An election analyst might recognise changes in regional voting behaviour, while an economist follows employment figures and another observer monitors campaign organisation. Their conclusions reflect different fragments of the same uncertain picture.
Prediction markets allow these assessments to influence a common price. Participants respond to existing valuations, incorporating information that other contributors may not possess.
The price attached to a bet on a sporting result provides a familiar example. Such markets are listed by betting sites, although bookmaker margins mean their quoted prices should not be interpreted as precise probabilities.
Market aggregation differs from a simple opinion poll. A survey records stated preferences or expectations; a trading market produces prices through transactions, with greater trading activity or capital potentially giving some participants more influence.
That mechanism can incorporate information quickly, but it does not automatically reward accuracy. Participants may face unequal resources, limited liquidity or incentives unrelated to forecasting precision.
Its informational value ultimately depends on the quality and independence of the assessments entering the market.
Why a correct prediction is not necessarily a good forecast
Forecasting performance cannot be evaluated solely by comparing a prediction with one eventual result.
Suppose an event receives an estimated probability of 70%. If it fails to occur, that outcome remains compatible with the original assessment, which explicitly allowed a 30% chance of failure.
A more rigorous assessment examines calibration: whether events assigned similar probabilities occur at approximately the corresponding frequency over repeated observations.
A well-calibrated set of 70% forecasts would therefore describe events occurring roughly seven times in ten. Calibration is not the only criterion, however. A forecast must also distinguish relatively likely outcomes from unlikely ones to provide useful information.
The 2004 paper explored how market prices could reveal expected outcomes and distributions of uncertainty. A contract predicting the winner of an election conveys different information from one estimating the candidate's vote share.
Without recognising these differences, comparisons between forecasting methods can become misleading.
The conditions under which crowds get it wrong
The informational advantages of aggregation weaken when participants share the same mistaken assumptions.
Herding illustrates the problem. An established market consensus can influence subsequent assessments, encouraging participants to repeat existing expectations rather than contribute independent evidence. Apparent agreement may conceal considerable informational dependence.
Thin trading creates another limitation. When participation is restricted or transactions are infrequent, quoted prices may respond poorly to new developments.
Wolfers and Zitzewitz also examined favourite–longshot bias, a recurring pattern in wagering markets in which low-probability outcomes can be priced too generously relative to their observed frequencies.
Such irregularities complicate the assumption that a market price represents an unbiased probability. Trading costs, market rules, unequal access to information and behavioural tendencies can all affect valuations.
The number of participants, considered alone, is consequently a poor measure of forecast reliability.
What collective forecasting changes about expert judgement
The research offers a framework for evaluating conflicting assessments in business, public policy and economic planning.
Expert opinion carries greater informational value when supported by independently obtained evidence and a documented forecasting record. Agreement among several commentators means less when their conclusions originate from the same assumptions or sources.
Markets introduce another form of evidence, but their prices require scrutiny of participation, incentives and historical accuracy. Neither professional credentials nor numerical consensus can establish that a forecast is reliable without examining how it was produced.
Specialists remain essential for explaining causal relationships, identifying structural changes and assessing circumstances poorly represented in historical data.
Wolfers and Zitzewitz showed that aggregated expectations can outperform established forecasting benchmarks. The more consequential finding concerns the conditions behind that performance: varied information, functioning mechanisms for incorporating it and assessments tested against observed results.
Betting carries financial risk. Market prices are uncertain estimates, not guarantees. Gambling is restricted to adults and should remain within personal financial limits.
