Forecast Backtesting
Allows you to compare historical forecasts for time periods over which actual data is collected and quantify the deviation between historical and forecasted trends. This lets you easily test the accuracy of forecasted trends and gain confidence in forecasts by allowing you to:
- Assess the sensitivity of forecasts to training range selection
- Assess the sensitivity of forecasts to outliers
- Assess the accuracy of longer-range forecasts (longer than 1/2 training period)
Figure 3-4 Forecast Backtesting UI
You perform Forecast Backtesting via Capacity Planning (CPU, Storage, Memory, I/O) for database and host resources. Immediately below the Trend & Forecast chart is Forecast Settings. Here, you set the Forecast Period (how far in the future you want to predict a trend) as well as the Training Period (the time period for which historical data is used to forecast a trend). Changes made to Forecast Settings are immediately reflected in the Trend & Forecast chart.
You can move, expand, or contract the Training Period selector to isolate the exact historical data you determine to be the most predictive for accurate forecasting. For example, you may want to exclude time periods that contain anomalous data spikes, which would throw off predictive analysis and result in higher percentage deviations between actual and predicted trends.
The forecast period begins immediately after the training period ends. More than just allowing you to set when the forecast should begin, the selector lets you begin the forecast period at a point in the past (backtesting), thus allowing you test how accurate the forecast will be based on how accurately it predicts a known historical trend. By default, Ops Insights creates a forecast period of 1/2 of the training period. You can set this to a specific number of days (45, 90, 180, and 365).
To get a quantitative look at how accurate the forecast is, hover your cursor over Average Deviation Values in the Forecast Settings pane to view the average percentage deviation between forecast and actual data.
To clear all forecast settings and start over, click Reset.