Results from evaluation of three commercial off-the-shelf face recognition systems on Chokepoint dataset


  1. Granger, E.
  2. Gorodnichy, D.O.
  3. Bergeron, J-P.
  4. Bissessar, D.
  5. Choy, E.
  6. Khreic, W.
  7. Radtke, P.
Corporate Authors
Defence Research and Development Canada, Centre for Security Science, Ottawa ON (CAN);Canada Border Services Agency, Ottawa ON (CAN);Quebec Univ, Montreal QUE (CAN)
This report is a supplement to Division Report 2014-27 (TR) ’‘Evaluation methodology for face recognition technology in video surveillance applications” (by E. Granger and D. Gorodnichy) [1]. It presents complete evaluation results of three Commercial Off-The-Shelf (COTS) Face Recognition (FR) systems: Cognitec, PittPatt and Neurotechnology — obtained on the Chokepoint public data-set using the multi-level evaluation methodology introduced in the previous report. Four levels of details are examined according to the methodology for each target person from the Checkpoint data-set: Level 0 or score-based analysis illustrates the probability distribution of the genuine scores against that of the impostors at different face resolutions, which visually illustrates the discrimination power of the system for each target individual. Level 1 or transaction-based analysis provides the averaged description of the system in terms of false positive and false negative rates aggregated over all transactions, expressed using Receiver Operative Curves (ROC), Detection Error Trade-off (DET), and Precision-Recall Operating Characteristic (PROC) curves. Level 2 or subject-based analysis describes the performance of the system using the-so-called “Doddington’s Zoo” categorization of individuals, which detects whether an individual belongs to an easier or a harder classes of people that the system is able to recognize [2, 3]. Finally, Level 3 or temporal analysis allows one to examine
video-surveillance;face recognition in video;instant face recognition;watchlist screening;biometrics;reliability;performance evaluation
Report Number
DRDC-RDDC-2014-C247 — Contract Report
Date of publication
01 Sep 2014
Number of Pages
Electronic Document(PDF)

Permanent link

Document 1 of 1

Date modified: