How robust are deep object detectors to variability in ground truth bounding boxes? experiments for target recognition in infrared imagery

Evan A. Stump, Francisco Reveriano, Leslie M. Collins, Jordan M. Malof

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

In this work we consider the problem of developing deep learning models - such as convolutional neural networks (CNNs) - for automatic target detection (ATD) in infrared (IR) imagery. CNN-based ATD systems must be trained to recognize objects using bounding box (BB) annotations generated by human annotators. We hypothesize that individual annotators may exhibit different biases and/or variability in the characteristics of their BB annotations. Similarly, computer-aided annotation methods may also introduce different types of variability into the BBs. In this work we investigate the impact of BB variability on the behavior and detection performance of CNNs trained using them. We consider two specific BB characteristics here: the center-point, and the overall scale of BBs (with respect to the visual extent of the targets they label). We systematically vary the bias or variance of these characteristics within a large training dataset of IR imagery, and then evaluate the performance on the resulting trained CNN models. Our results indicate that biases in these BB characteristics do not impact performance, but will cause the CNN to mirror the biases in its BB predictions. In contrast, variance in these BB characteristics substantially degrades performance, suggesting care should be taken to reduce variance in the BBs.

Original languageEnglish
Title of host publicationAutomatic Target Recognition XXX
EditorsRiad I. Hammoud, Timothy L. Overman, Abhijit Mahalanobis
PublisherSPIE
ISBN (Electronic)9781510635654
DOIs
StatePublished - 2020
EventAutomatic Target Recognition XXX 2020 - Virtual, Online, United States
Duration: Apr 27 2020May 8 2020

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume11394
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceAutomatic Target Recognition XXX 2020
Country/TerritoryUnited States
CityVirtual, Online
Period04/27/2005/8/20

Keywords

  • Automatic target recognition
  • Convolutional neural network
  • Deep learning
  • Object detection
  • infrared

Fingerprint

Dive into the research topics of 'How robust are deep object detectors to variability in ground truth bounding boxes? experiments for target recognition in infrared imagery'. Together they form a unique fingerprint.

Cite this