SAP IBP select forecasting algorithm with minimum positive error instead of min negative error?

[former_member856826]

Hello Consultants,

Why does the SAP IBP system select the forecasting algorithm with the minimum positive error instead of the algorithm with the overall lowest error, even when some algorithms have negative errors and others have positive errors, as indicated in the downloaded log file?

[richardlewis]

Hi Sumeet,

In your forecast profile, you have selected MPE (Mean Percentage Error) as the error measure to be used for bestfit selection. The MPE will show the errors in percentage terms as positive or negative, which indicates whether each model on average, is over (positive) or under (negative) forecasting over the error calculation period.

For MPE, the bestfit will pick the error which is the smallest percentage (closest to zero), regardless of whether it is positive or negative).This is the lowest overall error.

If you don't want to see the positive or negative in your bestfit error logs, consider using MAPE or WMAPE to show the absolute error values.

[lev_degtyarov]

Hi Sumeet,

just 50cent from my side. I saw you used MPE as error measure in your best-fit model. I just wonder why you decided on MPE. Because from data science community it is non usual, however still possible, to use MPE as metric to fine-tune you forecasting model.

Best practice is:

- MAPE (some times MPE) is used for reporting and dashboarding, to show forecast accuracy to the users

- RMSE or MAE (in SAP we call it also MAD) are used to fine tune hyper parameters in forecasting model, select the best model

- BIC, AIK - these one is for ARIMA class models

The pros and cons of MAPE, MAE, MAD, RMSE are know, you can google it an check youtube.

BR,

Lev