Self-Supervised Encoders Are Better Transfer Learners in Remote Sensing Applications

Zachary D. Calhoun, Saad Lahrichi, Simiao Ren, Jordan M. Malof, Kyle Bradbury

Research output: Contribution to journalArticlepeer-review

Abstract

Transfer learning has been shown to be an effective method for achieving high-performance models when applying deep learning to remote sensing data. Recent research has demonstrated that representations learned through self-supervision transfer better than representations learned on supervised classification tasks. However, little research has focused explicitly on applying self-supervised encoders to remote sensing tasks. Using three diverse remote sensing datasets, we compared the performance of encoders pre-trained through both supervision and self-supervision on ImageNet, then fine-tuned on a final remote sensing task. Furthermore, we explored whether performance benefited from further pre-training on remote sensing data. Our experiments used SwAV due to its comparably lower computational requirements, as this method would prove most easily replicable by practitioners. We show that an encoder pre-trained on ImageNet using self-supervision transfers better than one pre-trained using supervision on three diverse remote sensing applications. Moreover, self-supervision on the target data alone as a pre-training step seldom boosts performance beyond this transferred encoder. We attribute this inefficacy to the lower diversity and size of remote sensing datasets, compared to ImageNet. In conclusion, we recommend that researchers use self-supervised representations for transfer learning on remote sensing data and that future research should focus on ways to increase performance further using self-supervision.

Original languageEnglish
Article number5500
JournalRemote Sensing
Volume14
Issue number21
DOIs
StatePublished - Nov 2022

Keywords

  • SwAV
  • computer vision
  • domain adaptation
  • machine learning
  • satellite imagery
  • self-supervision
  • semantic segmentation
  • transfer learning

Fingerprint

Dive into the research topics of 'Self-Supervised Encoders Are Better Transfer Learners in Remote Sensing Applications'. Together they form a unique fingerprint.

Cite this