Systematic Prediction errors in O&G project and portfolio selection

Abstract

The normal rule for choosing between alternatives in a decision situation is to select the one with the maximum estimated value. Due to uncertainty in the estimates (prediction errors), the mere fact of choosing the maximum induces a systematic bias that guarantees that, over repeated decisions, less than the estimated expected value will be realized. Although some "proof-of- concept" instances of this behaviour (variously termed Post-decision Surprise, the Optimizer's Curse, Inevitable Disappointment) have been reported, it is not well-publicised in the O&G world and its importance/relevance in realistic O&G decision situations has not been thoroughly assessed. This paper places the effect in a more-widely applicable theoretical context and reports a systematic study that explores its impact on typical O&G decisions, using NPV as the value measure. Three typical situations, with different characteristics, are identified for investigation: intra-project alternative selection, using a Max[NPV] decision criterion; project "go"/"no go" decisions, using an NPV > 0 decision criterion; constrained portfolio selection, using a Max[NPV/I] criterion, subject to a budget limit. Sensitivity analysis of the magnitude of the effect is carried out in each case. We conclude that, whilst the effect is real and its magnitude may be large in some situations, it was of order 2% and 10% respectively, for the project and portfolio cases we analyzed. Further, the real prediction error, defined as expected difference between the true values of the selected alternative and the genuinely best alternative, was about half the above values. Given the range of other sources of prediction errors, plus the fact that its impact may be reduced due to corrective decisions as a project is executed, it may not, in practice, be as significant as previously suggested. Copyright 2008, Society of Petroleum Engineers.