文档章节

Land Cover Classification Using the Geo AI

openthings
 openthings
发布于 2018/04/27 14:44
字数 1069
阅读 126
收藏 1

Pixel-Level Land Cover Classification Using the Geo AI Data Science Virtual Machine and Batch AI

March 12, 2018 by ML Blog Team 

This post was authored by Mary Wahl, Kolya Malkin, Siyu Yang, Patrick Flickinger, Wee Hyong Tok, Lucas Joppa, and Nebojsa Jojic, representing the Microsoft Research and AI for Earth teams.

Last week Microsoft launched the Geo AI Data Science Virtual Machine (DSVM), an Azure VM type specially tailored to data scientists and analysts that manage geospatial data. To support the Geo AI DSVM launch, we are sharing sample code and methods for our joint land cover mapping project with the Chesapeake Conservancy and ESRI. We have used Microsoft’s Cognitive Toolkit (CNTK) to train a deep neural network-based semantic segmentation model that assigns land cover labels from aerial imagery. By reducing cost and speeding up land cover map construction, such models will enable finer-resolution timecourses to track processes like deforestation and urbanization. This blog post describes the motivation behind our work and the approach we’ve taken to land cover mapping. If you prefer to get started right away, please head straight to our GitHub repository to find our instructions and materials.

Motivation for the Land Cover Mapping Use Case

The Chesapeake Conservancy is a non-profit organization charged with monitoring natural resources in the Chesapeake Bay watershed, a >165,000 square kilometer region in the eastern U.S. extending from New York through Virginia. For a year and a half, the Chesapeake Conservancy spearheaded work with the Chesapeake Bay Program Partnership, WorldView Solutions, and University of Vermont Spatial Analysis Laboratory to meticulously assign each square meter in this region a land cover label such as water, grass/herbaceous, tree/forest, or barren/impervious surface. The resulting high-quality, high-resolution land cover map reflects 2013/2014 ground conditions and is a valuable resource already used by environmental scientists and conservation efforts throughout the region. If similar maps could be generated at frequent timepoints, the data would allow researchers to quantify trends such as deforestation, urbanization, and the impacts of climate change.

chesapeake_watershed_fullsize

Unfortunately, cost and time required for land cover map construction currently limit how frequently such maps can be produced. The Chesapeake Conservancy used several input data sources that had to be collected at the municipality, county, or state level, including LiDAR (to estimate height), multi-seasonal aerial imagery, and smaller-scale land use maps. In addition to the burden of assembling and curating these data sources, the Chesapeake Conservancy manually corrected errors because the rule-based classification workflow used for first-pass predictions of land cover labels was insufficiently accurate.

Our Land Cover Mapping Model

In collaboration with the Chesapeake Conservancy and ESRI, we at Microsoft helped train a deep neural network model to predict land cover from a single high-resolution aerial imagery data source collected nationwide at frequent intervals. We produced a neural network similar in architecture to Ronnenberger et al.’s U-net (2015), a commonly-used semantic segmentation model type. Our trained model’s predictions show excellent agreement with Chesapeake Conservancy’s land cover map. Some discrepancies reflect outdated or erroneous information in the original land cover map, including the small pond in the example below:

comparison_fullsize

After prototyping the training method on a single-GPU Azure Deep Learning Virtual Machine with a subset of available data, we scaled up training to a 148-GPU cluster using Azure Batch AI. This transition allowed us to reduce the average duration of each training epoch forty-fold, and further reduced runtime by allowing the complete ~1 TB training dataset to be stored in memory rather than repeatedly accessed from disk.

epoch_duration_scaling

We also addressed issues common with geospatial datasets such as making efficient use of very large, paired training image files (typical size: ~1 GB), label imbalance, and accommodating partial missingness of labels. Microsoft’s Cognitive Toolkit (CNTK) provided the flexibility needed to provide a custom data import method and loss function, while also simplifying the transition from single-process to distributed training.

Training and Applying the Model

Our pixel-level land classification tutorial contains both Jupyter notebooks for training the model on a single Geo AI DSVM, as well as instructions for training the model at scale using Batch AI. (You’re also welcome to adapt these files and methods for other types of computers, but you may find that you need to install some dependencies and ensure you have adequate hardware.) If you deploy a Geo AI DSVM, you’ll find copies of the Jupyter notebooks for this tutorial pre-installed in the “notebooks” folder under the user account’s home directory. Geo AI DSVMs can be created with 1, 2, or 4 NVIDIA K80 GPUs using VM sizes named NC6, NC12, and NC24, respectively: if you opt for multiple GPUs, you’ll also be able to test out the speed-up afforded by data-parallel distributed training.

Our Geo AI DSVM notebooks culminate by surfacing our trained model inside ESRI’s ArcGIS Pro, which comes pre-installed on the Geo AI DSVM. For researchers and analysts who work frequently with geospatial data, the ArcGIS software suite will be a familiar tool: it’s often used to create maps from raster or vector data, and to perform analyses that make use of spatial information. ArcGIS Pro is also a useful visualization tool: information can be computed and displayed in real-time while navigating through maps. ESRI’s Hua Wei has developed a custom raster function that can be used to call our land cover classification model in real-time and display the results alongside the original imagery and true labels: the code and sample ArcGIS project he developed, along with instructions on how to use them, are provided in our tutorial. While “ground-truth” labels are only available in the Chesapeake Bay watershed, you’ll be able to see the model’s predictions for regions throughout the continental U.S. in real-time.

arcgispro_finished_screenshot

Next Steps

We hope you enjoy our pixel-level land classification tutorial! We are very grateful to the Chesapeake Conservancy and ESRI for their help throughout this project and for allowing their data, trained models, and code to be shared here.

Please visit the following pages to learn more:

See our related example on embarrassingly-parallel classification of aerial images.

Mary, Kolya, Siyu, Patrick, Wee Hyong, Lucas & Nebojsa

本文转载自:https://blogs.technet.microsoft.com/machinelearning/2018/03/12/pixel-level-land-cover-classification

openthings
粉丝 325
博文 1140
码字总数 689435
作品 1
东城
架构师
私信 提问
Kaggle winner 方案简介 | Understanding the Amazon from Space: 1st place

Below is a brief introduction of the 1st place winner solution to the competition : Understanding the Amazon from Space The target of this competition is to better track and und......

不会停的蜗牛
2017/11/21
0
0
2018 AI、机器学习、深度学习与 Tensorflow 相关优秀书籍、课程、示例链接集锦

DataScienceAI Book Links | 机器学习、深度学习与自然语言处理领域推荐的书籍列表 人工智能、深度学习与 Tensorflow 相关书籍、课程、示例列表是笔者 Awesome Links 系列的一部分;对于其他...

王下邀月熊
2018/05/21
0
0
GIS及其相关期刊介绍

A list of relevant GIS journals We are curating a list of journals that we consider when submitting a GIS manuscript. Some of the journals listed below are not squarely GIS jour......

geowangsh
2016/02/03
51
0
Machine Learning 资讯 | 基于深度学习识别姑息治疗患者

“Hiring a Machine Learning engineer or Data Scientist in Silicon Valley is becoming like hiring a professional athlete. That’s how demanding it is” — The New York Times 基......

RiboseYim
2018/02/24
230
0
机器学习相关的网站

机器学习网站整理 机器学习相关网站整理: 北京大学视觉与听觉信息处理实验室 北京邮电大学模式识别与智能系统学科 复旦大学智能信息处理开放实验室 IEEE Computer Society北京映象站点 计算...

机器学习天天向上
2014/01/05
2K
3

没有更多内容

加载失败,请刷新页面

加载更多

Mybatis Plus删除

/** @author beth @data 2019-10-17 00:30 */ @RunWith(SpringRunner.class) @SpringBootTest public class DeleteTest { @Autowired private UserInfoMapper userInfoMapper; /** 根据id删除......

一个yuanbeth
今天
4
0
总结

一、设计模式 简单工厂:一个简单而且比较杂的工厂,可以创建任何对象给你 复杂工厂:先创建一种基础类型的工厂接口,然后各自集成实现这个接口,但是每个工厂都是这个基础类的扩展分类,spr...

BobwithB
今天
4
0
java内存模型

前言 Java作为一种面向对象的,跨平台语言,其对象、内存等一直是比较难的知识点。而且很多概念的名称看起来又那么相似,很多人会傻傻分不清楚。比如本文我们要讨论的JVM内存结构、Java内存模...

ls_cherish
今天
4
0
友元函数强制转换

友元函数强制转换 p522

天王盖地虎626
昨天
5
0
js中实现页面跳转(返回前一页、后一页)

本文转载于:专业的前端网站➸js中实现页面跳转(返回前一页、后一页) 一:JS 重载页面,本地刷新,返回上一页 复制代码代码如下: <a href="javascript:history.go(-1)">返回上一页</a> <a h...

前端老手
昨天
5
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部