Soft-masks guided faster region-based convolutional neural network for domain adaptation in wind turbine detection

Yang Xu, Xiong Luo, Manman Yuan, Bohao Huang, Jordan M. Malof

Research output: Contribution to journalArticlepeer-review

Abstract

Wind turbine generator system plays a fundamental role in electricity generation in industry 4.0, and wind turbines are usually distributed separately and in poor locations. Unmanned Aerial Vehicles (UAV) which could overcome the above challenges are deployed to collect photographs of wind turbines, could be used for predictive maintenance of wind turbines and energy management. However, identifying meaningful information from huge amounts of photographs taken by drones is a challenging task due to various scales, different viewpoints, and tedious manual annotation. Besides, deep neural networks (DNN) are dominant in object detection, and training DNN requires large numbers of accurately labeled training data, and manual data annotation is tedious, inefficient, and error-prone. Considering these issues, we generate a synthetic UAV-taken dataset of wind turbines, which provides RGB images, target bounding boxes, and precise pixel annotations as well. But directly transferring the model trained on the synthetic dataset to the real dataset may lead to poor performance due to domain shifts (or domain gaps). The predominant approaches to alleviate the domain discrepancy are adversarial feature learning strategies, which focus on feature alignment for style (e.g., color, texture, illumination, etc.) gaps without considering the content (e.g., densities, backgrounds, and layout scenes) gaps. In this study, we scrutinize the real UAV-taken imagery of wind turbines and develop a synthetic generation method trying to simulate the real ones from the aspects of style and content. Besides, we propose a novel soft-masks guided faster region-based convolutional neural network (SMG Faster R-CNN) for domain adaptation in wind turbine detection, where the soft masks help to extract highly object-related features and suppress domain-specific features. We evaluate the accuracy of SMG Faster R-CNN on the wind turbine dataset and demonstrate the effectiveness of our approach compared with some prevalent object detection models and some adversarial DA models.

Original languageEnglish
Article number1083005
JournalFrontiers in Energy Research
Volume10
DOIs
StatePublished - Jan 30 2023

Keywords

  • domain adaptation (DA)
  • faster region-based convolutional neural network (Faster R-CNN)
  • soft-masks
  • synthetic data
  • wind turbine detection

Fingerprint

Dive into the research topics of 'Soft-masks guided faster region-based convolutional neural network for domain adaptation in wind turbine detection'. Together they form a unique fingerprint.

Cite this