The RAE FM[Triage] Protocol: Selection Among Forecasting Model Candidates

Abstract

Context. The Relative Absolute Error [RAE], proposed by Theil (1958) and calibrated by Armstrong & Collopy (1992), for a forecasting context, is a benchmarking-measure used by many forecasters. Experientially, we find that, due to its composition as: The ratio of two Absolute Percentage Error [APE]-profiles, the RAE is often confusing to Decision-Makers wishing to use the RAE to select among Forecasting Model [FM]-Candidates—i.e., FM-Triage. Focus. In consultations, we often advise forecasters to Keep It Simple and initialize their FM-triage by using RAE-screening for: The two most basic and effective FMs: {The OLS-R Time Series [OLSR] & The Random Walk [RW]}. Our tacit assumption is: As there is NO FM that is the best overall for Panels that require forecasts for multiple extrapolations, initially examine which of these two Basic Models emerges as Relatively the Best as measured by their RAE-Profiles. Rationale. Over the years, both in consultations and for academic presentations, it has been clear that the Best FMs depend upon (i): The Nature of the Data, and (ii) The Nature of the Extrapolations. Focus. We offer, for the first time, an instructive RAE-triage-analysis of: {The OLSR & The RW} FMs for: (i) A Set of Panels randomly sampled from the S&P₅₀₀, then (ii) Grouped by their Coefficient of Determination [CoD], for (iii) Forecasts over three Extrapolations. Results. Indeed, the nature of the presentation of the RAE-FM-profiles can be a source of confusion. Thus, we offer a multi-Stage RAE FM[Triage] Protocol to simplify the FM-selection procedure. We found that: (i) The Nature of the CoD-Dataset & Extrapolations likely impact the nature of the FM that has the best RAE-profile, and (ii) not infrequently, both the OLSR & RW need to be used to refine the forecasting acuity.