For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-babenko For more information about embedded vision, please visit: http://www.embedded-vision.com Boris Babenko, Senior Software Engineer at Orbital Insight, presents the "Using Satellites to Extract Insights on the Ground" tutorial at the May 2017 Embedded Vision Summit. Satellites are great for seeing the world at scale, but analyzing petabytes of images can be extremely time-consuming for humans alone. This is why machine vision is a perfect complement for satellite imagery: providing human-quality analysis at inhuman scale. This talk explains how different types of satellites are best suited to different types of observation and discusses applications of machine vision. Babenko looks at how this technology was able to not only detect previously unknown oil tanks in China, but also estimate how much oil the country stores in these tanks. He also looks at how simple metrics like car density and building density can be applied to both commercial and humanitarian applications. The talk concludes by examining the possibilities for the future as our ability to both take and process images continues to grow.

1. Copyright © 2017 Orbital Insight 1
Using Satellites to Extract Insights
on the Ground
Dr. Boris Babenko, Orbital Insight
May 2017

2. Copyright © 2017 Orbital Insight 2
• Founded in 2013
• Our mission: to understand socio-economic trends on a global scale,
using aerial imagery
• Our technology: deep learning and computer vision + data science
• Our partners: Airbus, DigitalGlobe, Planet, ImageSat, MDA Corporation,
Urthecast, USGS Landsat
• Our customers: >70 asset management firms, several U.S. government
agencies, 2 global nonprofits
About Orbital Insight

3. Copyright © 2017 Orbital Insight 3
Background: Why Space Now?

4. Copyright © 2017 Orbital Insight 4
• Building and launching a satellite is
cheaper than ever before
• Artificial intelligence has made great
advances
• Satellite imagery: a previously untapped
resource outside of government
The Perfect Combination
Image Credit: NASA

5. Copyright © 2017 Orbital Insight 5
• In a few years, you’d need every person in New York City to spend all
day, every day, looking at photos in order to have humans lay eyes on
each satellite image being generated daily.
Why Do We Need Artificial Intelligence?
Image Credit: matheuslotero, Flickr, CC Attribution license

6. Copyright © 2017 Orbital Insight 6
Satellites: Not One-Size-Fits-All

7. Copyright © 2017 Orbital Insight 7
• Bigger satellites
• More expensive to make & launch (~5,000 lb),
so fewer in orbit
• As a result, less frequent imagery
• Higher resolution photos (e.g., 0.5 m)
• Smaller satellites (nanosats, cubesats, etc.)
• Less expensive to make & launch (~10 lb), so
more in orbit
• As a result, more frequent imagery
• Lower resolution photos (e.g., 3 m)
Size, Frequency and Detail
Photo Credit: DigitalGlobe
Photo Credit: Planet

8. Copyright © 2017 Orbital Insight 8
What Can We Measure?

9. Copyright © 2017 Orbital Insight 9
• Convolutional neural network
trained to detect cars in 0.5 m
imagery – cars are only a few
pixels in size
• Track hundreds of retail chains,
malls, etc.
• Parking lot traffic correlates
with sales
Retail Sales Forecasting
Photo Credit: Orbital Insight/satellite imagery: DigitalGlobe

10. Copyright © 2017 Orbital Insight 10
0
0.05
0.1
0.15
0.2
0.25
0.3
$0
$100
$200
$300
$400
$500
$600
$700
$800
1/1/10 7/1/10 1/1/11 7/1/11 1/1/12 7/1/12 1/1/13 7/1/13 1/1/14 7/1/14 1/1/15 7/1/15 1/1/16 7/1/16
CMGIndexedTraffic
CMGStockPrice
CMG Stock Price CMG Indexed Traffic
Street consensus
estimates fail to pickup
ongoing slowdown in
traffic, stock corrects -40%
after six-month
dislocation
Orbital Insight
uncovers a
slowdown in
traffic, missed
by consensus
Stock declines -20%
after nine-month
dislocation from traffic
patterns
Stock declines -45%
to more closely
correlate to the 12
month deterioration
in traffic
Orbital Insight
observes traffic peak
in early 2015, stock
continues to deviate
from underlying
deterioration in
traffic
Orbital Insight uncovers
deterioration in traffic,
dislocation from Wall Street
expectations.
Helping Uncover Dislocations to Street
Expectations

11. Copyright © 2017 Orbital Insight 11
-0.25
-0.2
-0.15
-0.1
-0.05
0
0.05
$0
$5
$10
$15
$20
$25
$30
$35
$40
$45
$50
1/1/10 7/1/10 1/1/11 7/1/11 1/1/12 7/1/12 1/1/13 7/1/13 1/1/14 7/1/14 1/1/15 7/1/15 1/1/16 7/1/16
JCPIndexedTraffic
JCPStockPrice
JCP Stock Price JCP Indexed Traffic
JCP stock price performance mirrors traffic after three year
period
Uncovering Early Short Opportunities

12. Copyright © 2017 Orbital Insight 12
What else can be done with a car detector?
Nanjing, China
• Detect cars across an entire city
• Proxy for: urban development,
population, income, gasoline
demand

13. Copyright © 2017 Orbital Insight 13
• Estimating how much oil
is in each tank
• Tracking 20,000+ tanks
worldwide
Tracking worldwide crude oil inventory
Photo Credit: Orbital Insight/satellite imagery: DigitalGlobe, Planet
0.5 m imagery 3 m imagery

14. Copyright © 2017 Orbital Insight 14
Tracking worldwide crude oil inventory

15. Copyright © 2017 Orbital Insight 15
Previously known
Found by Orbital Insight
• Ran CNN across all China to find
unknown oil tank farms
Building a complete catalog of tanks

16. Copyright © 2017 Orbital Insight 16
• Never enough data
• Trade-off between spatial resolution and temporal frequency
• Clouds
• Can’t count cars if they’re covered by clouds
• Cloud computing
• Computational resources seem infinite and
inexpensive… until you start using GPUs
• Recruiting engineers (we’re hiring!)
Challenges
Photo Credit: Orbital Insight/satellite imagery: DigitalGlobe, Planet, USGS

17. Copyright © 2017 Orbital Insight 17
Future Possibilities

18. Copyright © 2017 Orbital Insight 18
Railcar storage Landuse classification
Iron ore stockpiles Aircraft monitoring Heavy industry
Photo Credit: Orbital Insight/satellite imagery: DigitalGlobe, Planet, USGS
Other applications

19. Copyright © 2017 Orbital Insight 19
• More verticals
• Infrastructure and asset monitoring
• Agriculture
• Integrating more data
• More satellite imagery
• Beyond optical: SAR
• Beyond satellites: UAV/Drone imagery
• Beyond imagery: AIS, other GIS datasets
Where We Go From Here

20. Copyright © 2017 Orbital Insight 20
• The Science Behind the Signal: Tracking Unknown Oil Tanks Around
the World, Orbital Insight’s blog
• From the Macroscope: Home Improvement Stores End 2015 With
a Whimper, Orbital Insight’s blog
• Leveraging Commercial Applications to Help the World Bank
Map Poverty, Orbital Insight’s blog
• “Agricultural Crop Health Analysis” in Deep Learning Use Cases for
Computer Vision, by Tractica, Embedded Vision Alliance website
Resources