INCOMMANDS TDP – Measures of performance for the threat evaluation capability

PDF

Authors
  1. Frini, A.
  2. Benaskeur, A.
Corporate Authors
Defence R&D Canada - Valcartier, Valcartier QUE (CAN)
Abstract
In this document, we are concerned with the Threat Evaluation (TE) capability, developed for the INCOMMANDS TDP to support the operator involved in the shipboard TE process. In particular, we focus on evaluating the recommendations provided by this capability as well as evaluating the final TE decisions made by the operator when using this capability. A conceptual model explaining how these evaluations will be done is proposed. First, we propose a set of measures of performance (MoPs) for the assessment of the quality of the recommendations provided by the TE capability. These measures are: measures of accuracy and agreement, measures of timeliness, sensitivity analysis, input degradation analysis and anytime behaviour analysis. Second, we consider that the assessment of the level of cognitive activity that the human operator is experiencing is important. This evaluation will assess if higher quality of decision making is reached while the operator is overwhelmed or not. To do so, we propose the use of the following human factors metrics: situation awareness, workload and task relevant behaviour. Third, the quality of the TE decisions made by the operator (when using the capability) will be measured using complementary MoPs such as accuracy, timeliness and consistency. When analyzed together, these measures provide converging evidence that decision making quality has either increased or decreased.

Il y a un résumé en français ici.

Report Number
DRDC-VALCARTIER-TM-2009-240 — Technical Memorandum
Date of publication
01 Dec 2009
Number of Pages
68
DSTKIM No
CA033593
CANDIS No
532950
Format(s):
Electronic Document(PDF)

Permanent link

Document 1 of 1

Date modified: