The key idea
A baseline is the simple forecast a more complex model must improve upon. It might use recent average demand or a relevant seasonal comparison. Without a baseline, an impressive dashboard can show plausible numbers without demonstrating additional value. The comparison should use the same information dates, titles and decision horizon.
Forecast evaluation is described in the forecasting reference. The numerical comparison is an original illustrative example. [1]
Choose a reference that matches the operation
For a relatively steady title, recent average daily demand can be a useful starting reference. A strongly seasonal title may require a comparison with a similar period instead. Document the rule before evaluating the advanced model. Do not change the reference each time a new result appears. Also record the minimum history needed and the fallback for a title with no usable observations, so missing inputs do not become invisible advantages for either model.
Evaluate decisions, not only the forecast line
A forecast can have lower average error while still performing poorly on the expensive situations. Compare shortages, excess stock and the number of unnecessary production changes as well as a statistical error measure. Keep the replenishment rules consistent across both forecasts. Otherwise a change in stock policy can be mistaken for an improvement in prediction. Report results by title group so a few popular books do not hide weak performance across the backlist.
Worked example · illustrative assumptions
Suppose a baseline forecasts ten copies during each production lead time. An advanced model forecasts eight, twelve and nine over three trials; actual demand is nine, eleven and ten. The baseline absolute errors are one, one and zero. The advanced model errors are one, one and one. In this small example, the advanced model is not better. More sophisticated inputs are not a reason to promote it without stronger evidence.
Put it into practice
- Fix the baseline definition and evaluation dates before comparing models.
- Use identical stock policies when comparing operational results.
- Keep the simpler method when the new model has not demonstrated value.
Watch out for
Do not compare a new model tested on easy, high volume titles with a baseline tested on the whole catalog. Unequal samples can make an unsuccessful change look like progress.
Does a baseline mean the service is not using AI?
No. A baseline is a quality control. AI can research signals and propose forecasts while the operational system retains the best supported decision method for each title group.
Sources and further reading
- Hyndman and Athanasopoulos: The forecaster's toolbox. Baseline methods, evaluation and intervals; examples here are original.
AI assisted editorial content with original planning examples. Source links support the stated source notes; operational suggestions are guidance rather than guarantees. Provider rules and product coverage can change. Editorial approach · BooksAhead.ai connection status.