How generalizable is good judgment? – A multi-task, multi-benchmark study

PDF

Authors
  1. Mellers, B.A.
  2. Baker, J.D.
  3. Chen, E.
  4. Mandel, D.R.
  5. Tetlock, P.E.
Corporate Authors
Defence Research and Development Canada, Toronto Research Centre , Toronto ON (CAN)
Abstract
Good judgment is often gauged against two gold standards – coherence and correspondence. Judgments are coherent if they demonstrate consistency with the axioms of probability theory or propositional logic. Judgments are correspondent if they agree with ground truth. When gold standards are unavailable, silver standards such as consistency and discrimination can be used to evaluate judgment quality. Individuals are consistent if they assign similar judgments to comparable stimuli, and they discriminate if they assign different judgments to dissimilar stimuli. We ask whether “superforecasters”, individuals with noteworthy correspondence skills (see Mellers et al., 2014) show superior performance on laboratory tasks assessing other standards of good judgment. Results showed that superforecasters either tied or out-performed less correspondent forecasters and undergraduates with no forecasting experience on tests of consistency, discrimination, and coherence. While multifaceted, good judgment may be a more unified than concept than previously thought.
Keywords
Judgment;coherence;superforecasters
Report Number
DRDC-RDDC-2017-P053 — External Literature
Date of publication
01 Aug 2017
Number of Pages
17
Reprinted from
Judgement and Decision Making, Vol. 12. No.4, 2017, p. 369-381
DSTKIM No
CA045250
CANDIS No
805646
Format(s):
Electronic Document(PDF)

Permanent link

Document 1 of 1

Date modified: