TY - GEN
T1 - Do Deep Learning Models Generalize to Overhead Imagery from Novel Geographic Domains? the xGD Benchmark Problem
AU - Huang, Bohao
AU - Bradbury, Kyle
AU - Collins, Leslie M.
AU - Malof, Jordan M.
N1 - Publisher Copyright:
© 2020 IEEE.
PY - 2020/9/26
Y1 - 2020/9/26
N2 - Recently, Convolutional Neural Networks (CNNs) have demonstrated impressive performance on several visual recognition benchmark datasets utilizing overhead imagery. However, most of these analyses performed on benchmark datasets involve testing pre-trained CNNs on imagery that was collected over roughly the same locations as the training imagery. In this work we propose a benchmark problem - termed cross-geographical domain (xGD) adaptation - designed to evaluate the performance of CNNs in which they are tested on imagery collected over previously unseen geo-locations - a more challenging and practical scenario that we term cross-domain testing. We focus this work on building segmentation due to the availability of appropriate datasets. The results indicate that CNNs generalize poorly to data processed from geographic locations that were not present in training. Surprisingly, we found that larger models (pre-trained on ImageNet) generalize as well as small models in cross-domain testing, and sometimes better. This work provides the first comprehensive results for cross-domain recognition, raising awareness of this important problem. We hope that xGD can serve as a benchmark for future work; xGD uses publicly-available data, and we release our design details with this publication.
AB - Recently, Convolutional Neural Networks (CNNs) have demonstrated impressive performance on several visual recognition benchmark datasets utilizing overhead imagery. However, most of these analyses performed on benchmark datasets involve testing pre-trained CNNs on imagery that was collected over roughly the same locations as the training imagery. In this work we propose a benchmark problem - termed cross-geographical domain (xGD) adaptation - designed to evaluate the performance of CNNs in which they are tested on imagery collected over previously unseen geo-locations - a more challenging and practical scenario that we term cross-domain testing. We focus this work on building segmentation due to the availability of appropriate datasets. The results indicate that CNNs generalize poorly to data processed from geographic locations that were not present in training. Surprisingly, we found that larger models (pre-trained on ImageNet) generalize as well as small models in cross-domain testing, and sometimes better. This work provides the first comprehensive results for cross-domain recognition, raising awareness of this important problem. We hope that xGD can serve as a benchmark for future work; xGD uses publicly-available data, and we release our design details with this publication.
KW - building segmentation
KW - domain adaptation
KW - overhead imagery
KW - segmentation
UR - http://www.scopus.com/inward/record.url?scp=85101968015&partnerID=8YFLogxK
U2 - 10.1109/IGARSS39084.2020.9323080
DO - 10.1109/IGARSS39084.2020.9323080
M3 - Conference contribution
AN - SCOPUS:85101968015
T3 - International Geoscience and Remote Sensing Symposium (IGARSS)
SP - 1476
EP - 1479
BT - 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020
Y2 - 26 September 2020 through 2 October 2020
ER -