This planet is a big place filled with amazing and unusual things. Understanding every object, location, and action on this pale blue dot is an enormous challenge. As the world’s leading provider of high-resolution Earth imagery, data, and analysis, DigitalGlobe faces this challenge every day. They use automatic computer vision and machine learning where possible, but so far, the only true solution requires the most powerful information processing machine we know: the human brain. Scaling this solution to work on the trillions of satellite pixels collected by DigitalGlobe every day requires thousands of brains, all working in harmony. To address this, DigitalGlobe | Radiant's (now Radiant Solutions) Tomnod service uses Amazon Mechanical Turk, a crowdsourcing internet platform, to identify small objects appearing in large areas of new satellite imagery. Tomnod is heavily used for commercial and humanitarian purposes. In this session, you hear how Radiant Solutions uses crowdsourcing to help solve large-scale computer vision and machine learning problems.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
An Eye in the Sky
H o w Radi ant So l uti o ns Pr o c e sse s Sate l l i te
Im ag e r y wi th A I and A m azo n Me c h an i c al T ur k
K e v i n M c G e e – P r o d u c t i o n L e a d , R a d i a n t S o l u t i o n s
M C L 2 5 1
AWS re:INVENT
D e c e m b e r 1 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Radiant Solutions is built on the pedigree of
DigitalGlobe | Radiant and MDA Information Systems

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Maxar Technologies will leverage the power of
four leading space brands

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introduction
• Production lead—Radiant Solutions
• Machine learning team
Data
generation
AI
training
AI
integration

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introduction
• Production lead—Radiant Solutions
• Machine learning team
Data
generation
AI
training
AI
integration

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Session overview
• Overview: How we use machine learning and Mechanical Turk to process
satellite imagery
• Level of knowledge: Beginner
• Content:
• Problem scale
• Machine learning and imagery
• Tomnod
• Generating data with Mechanical Turk and Tomnod
• Closing

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Problem scale

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Problem scale
• We use DigitalGlobe imagery
• Five Earth observation satellites on orbit
• 3 million km2 of imagery taken every day
• Highest resolution commercially available
• Millions of objects of interest coming in every day
• Cars, trucks, aircraft, infrastructure, etc.

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Translation
“Increased shipping activity in this location”

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
“If we looked at the proliferation of the new satellites over time, and we
continue to do business the way we do, we’d have to hire two million more
imagery analysts.” –Scott Currie, NGA Mission Integration Director

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Machine learning and imagery

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Labeled
images
How it works—R&D
Neural
network
Model

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Labeled
images
How it works—Radiant Solutions
Neural
network
Model

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ML and ground imagery
• Image competitions
• ImageNet
• PASCAL VOC
• Microsoft COCO
• Industry examples
• Autonomous vehicles
• Facebook facial recognition
Bob

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ML and satellite imagery
• Started with classifiers
• Urban, Rural, Forest
• Now using detectors
• An object is located here on this
image
Water
Water
Water
Rural
Urban
Forest

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tomnod

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tomnod overview
• Tomnod is our crowdsourcing application
• Serves satellite imagery to the crowd
• Tools to tag the satellite imagery
• We need three things to make a good model:
1. Location
2. Object type
3. Lots of examples
+ +

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tomnod and Amazon Mechanical Turk
• We use Amazon Mechanical Turk to help scale this problem
• We need massive amounts of training data quickly, timely, and cost-
effectively
• Mechanical Turk was a good fit
+ = Meaningful
Data

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tomnod—Disaster Recovery

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Validation

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Validate—Options
Existing data Open source Model results
OpenStreetMap

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Generating data with Tomnod and
Mechanical Turk

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tomnod designed for Mechanical Turk use
Many examples provided Multiple coverage
1x
2x
3x
Crowd-rank algorithm
99% Confidence

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Mechanical Turk qualifications
Qualifications help Requesters find Workers best suited to their tasks
Example of Qualifications:
Requesters specify the Qualification
requirements for their tasks
Workers can see the Qualifications
required for a task, to determine
their eligibility

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Overall process
Create Validate Recreate

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Create

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Validate

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recreate

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Closing

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Closing
• Where are we now?
• Generated over one million labeled objects
• Where are we going?
• Models as a Service (MaaS)
• How we’ll get there

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Radiant Solutions
Other

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
K e v i n M c G e e – P r o d u c t i o n L e a d , R a d i a n t S o l u t i o n s
k e v i n . m c g e e @ d i g i t a l g l o b e . c o m