Deep Learning for Remote Sensing

A remarkable aspect of deep learning is that its neural networks are powerful learning machines that generalize to different domains. For example, semantic segmentation model performs the underlying task of classifying pixels, whether those pixels are photons from a camera mounted on a self-driving car, radiodensities from X-ray tubes in a CT scanner, or seismic reflections measured by a sensor array. The same model, trained on different data, can learn to perform tasks as varied as predicting oil fault lines, detecting cars, or predicting blood flow.

Recently, an exciting trend has emerged of applying deep learning to remote sensing—pixels measured from space. Multiple data science challenges were launched using satellite imagery, for building footprints, road networks, object detection, land use classification, sea lion counting, iceberg detection, tracking deforestation, feature detection, data fusion, and more. Algorithms are being used for additional applications such as monitoring green spaces or triaging disaster zones. The amount of data being collected is accelerating rapidly. For example, companies such as Planet Labs have satellite constellations capable of imaging the entirety of Earth once per day[1].

On one hand, deep learning lends itself well to remote sensing. Large swaths of imagery are collected, much of which are uninteresting and could benefit from automated filtering. Unlike in autonomous driving or other applications, change can occur gradually over long time scales that need patient monitoring. Computers have infinite patience.

Yet, remote sensing brings interesting research and application challenges. The default approach is to apply computer vision networks designed for natural scene datasets such as ImageNet or PASCALVOC. However, there are significant differences in this type of data (see Figure 1):

  • Object density and size span multiple orders of magnitude
  • Imagery are not just RGB, but can include up to dozens of bands. For example, the IEEE GRSS challenge includes data with 48 spectral bands.
  • Objects have rotational symmetry

Besides these fundamental differences, additional deployment challenges exist. From dealing with cloud cover and other missing data, to integrating imagery with multiple scales and observation angles, there are many exciting technical hurdles to realizing the vision of applying AI to extract insights about our home planet.

Here at the Intel AI Lab, we have several research projects to adapt existing models to this domain, and also explore novel architectures. For example, we have modified the popular Single Shot Detection (SSD) model for object detection to predict oriented bounding boxes. In another one of our projects, we developed multi-stream models that use attention mechanisms to select which spectral bands to attend to. We have applied these methods to identifying resources for lunar exploration from NASA’s Lunar Reconnaissance Orbiter (LRO). An Intel engineer recently won the Kaggle challenge for iceberg detection.

Even though new labeled datasets have been introduced in the last year, it’s not easy for data scientists who are not geospatial experts to get started with satellite imagery. Existing dataloaders do not work with GeoTIFF formats, especially for multi-band images. New tools are needed to extract the metadata and translate from coordinate systems to pixels. Images can be 20,000 by 20,000 pixels, stressing a server’s memory and I/O bandwidth. The community had started to come to introduce tools, such as the SpaceNet Utilities and rasterio, to attract a larger community of data scientists. At Intel AI DevCon from May 23-24, join us in a hands-on lab to learn how to use some of these tools to train a simple building detection model on satellite imagery in 10 minutes using an Intel® Xeon® Scalable Processor.

We are excited to work with the data science community to accelerate the use of deep learning to better understand our own planet.

Notices and Disclaimers

Intel, the Intel Logo, and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.

© Intel Corporation

[1] https://www.planet.com/pulse/planet-launches-satellite-constellation-to-image-the-whole-planet-daily/