Accuracy of Forecasts in Strategic Intelligence

PDF

Authors
  1. Mandel, D.R.
  2. Barnes, A.
Corporate Authors
Defence Research and Development Canada, Toronto Research Centre , Toronto ON (CAN)
Abstract
The accuracy of 1,514 strategic intelligence forecasts abstracted from intelligence reports was assessed. The results show that both discrimination and calibration of forecasts was very good. Discrimination was better for senior (versus junior) analysts and for easier (versus harder) forecasts. Miscalibration was mainly due to underconfidence such that analysts assigned more uncertainty than needed given their high level of discrimination. Underconfidence was more pronounced for harder (versus easier) forecasts and for forecasts deemed more (versus less) important for policy decision-making. In spite of the observed underconfidence, there was a paucity of forecasts in the least informative 0.4-0.6 probability range. Recalibrating the forecasts substantially reduced underconfidence. The findings offer cause for tempered optimism about the accuracy of strategic intelligence forecasts and indicate that intelligence producers aim to promote informativeness while avoiding overstatement.
Keywords
forecasting;accuracy;strategic intelligence
Report Number
DRDC-RDDC-2016-P72 — External Literature
Date of publication
01 Sep 2014
Number of Pages
21
DSTKIM No
CA042960
CANDIS No
804235
Format(s):
Electronic Document(PDF)

Permanent link

Document 1 of 1

Date modified: