Making Earth observation data available by using Amazon S3 is accelerating scientific discovery and enabling the creation of new products. Attend and learn how the scale and performance of Amazon S3 lets earth scientists, researchers, startups, and GIS professionals gather and analyse planetary-scale data without worrying about limitations of bandwidth, storage, memory, or processing power. Co-presented with support of the Australian Geoscience Data Cube collaboration, DigitalGlobe’s Geospatial Big Data Platform and the developer of the popular ObservedEarth mobile app.
Speakers:
Craig Lawton, Public Sector Solutions Architect, Amazon Web Services
Lachlan Hurst, Observed Earth
Matt Paget, Senior Experimental Scientist, CSIRO
Dan Getman, Digital Globe
2. What to Expect from the Session
• Open data on AWS and the advantages of using Amazon S3
• ObservedEarth Developer
• CSIRO & Australian Geoscience Data Cube
• DigitalGlobe & PSMA
3. Why Does AWS Care About Open Data?
Sharing data on AWS makes it accessible to a large and
growing community of researchers, entrepreneurs,
and enterprises who use the AWS cloud.
4. “…data must be organised, well-documented,
consistently formatted, and error free. Cleaning
the data is often the most taxing part of data
science, and is frequently 80% of the work.”
- Data Driven by DJ Patil and Hilary Mason
Undifferentiated Heavy Lifting
8. AWS Cloud Credits for Research
provide promotional AWS cloud credits
for anyone to conduct research using
Earth Observation data.
aws.amazon.com/earth/research-credits
13. First Came the Data…
Landsat 8
• Multispectral
• 30m resolution
• Global coverage
• Updated every 16 days
• Open data
Landsat on AWS
• 1 million scenes (Feb 2017)
• 800MB per scene
• Updated daily
• Data freely available on S3
Image credit: NASA, GSFC,
Landsat
14. … then the Idea
ObservedEarth aims to make Landsat data
accessible to everyone.
• Smartphone
• App with access to data
• Build in domain expertise
Little app for a lot of data
19. How Does it Work?
GPUs are great for processing raster data
• Landsat data downloaded from S3 read into textures
• ObservedEarth filters implemented as fragment shaders
20. Reception
Users
• 200000 scenes viewed
• 400GB data downloaded
Media
ObservedEarth: A Talk with its Creator - Q&A with NASA
#Landsaturated - Medium post by Joe Morrison
“. . .a brilliant little tool for
students or anyone interested
in the earth and remote
sensing. Thank you”
“Awesome!! Thank you for your
time. I'm a student, currently
working on landslide hazard ...I
have no words. It is beautiful!”
“Great to play with”
App Store reviews
24. Our Next Challenge
People expect access to EO data
and products
New EO satellite missions
Will require fundamental changes
to how EO data are analysed
Courtesy of Geoscience Australia
25. What are (EO) Data Cubes?
• Data Cube = Spatially aligned pixels
ready for analysis
• Proven concept by Geoscience
Australia (GA), the Australian Science
Agency (CSIRO) and a super-
computer facility (NCI)
• Shift in Paradigm ... Pixels vs Files
• Analysis Ready Data (ARD) ...
Reduce processing burden on users
• Supports integration of multiple
datasets.
• Multi-platform … PC, HPC, Cloud TIME
Open Source Software
https://github.com/opendatacube
26. Data Cubes and AWS
AWS for exploratory analysis
and market access
• Fast, scalable and accessible
• Address multi-platform criteria
New data structures for EO
• Aligned with S3 principles
• Analytics to take advantage of
I/O concurrency, built on new
low-level APIs
• Execution engine exploiting
the cloud
27. Earth Analytics Industry Hub
Public good component
using free satellite data
• A global network of
interoperable data cubes
Commercial component
• Business processes and engagement to sustain
science and data pipelines
“App Store” of analytics routines
• EO and geoscience researchers delivering to
minerals, agriculture and environment domains
Multi-platform
• Data cube applications work seamlessly across
Cloud, HPC and your desktop
28. Thank you
Matt Paget matt.paget@csiro.au
Rob Woodcock robert.woodcock@csiro.au
Australian Geoscience Data Cube and the
Open Data Cube Community
• Geoscience Australia
• CSIRO
• National Computational Infrastructure
• Committee on Earth Observing Satellites
• NASA and USGS
30. DigitalGlobe is…
The market leader in remote sensing.
IKONOS®
.82 meter
resolution
9 m CE90
QuickBird®
.65 meter
resolution
23 m CE90
WorldView-4®
.30 meter
resolution
<3.0 m CE90
WorldView-3®
.30 meter
resolution
<3.0 m CE90
WorldView-2®
.46 meter
resolution
<3.5 m CE90
GeoEye-1®
.46 meter
resolution
<3.5 m CE90
WorldView-1®
.50 meter
resolution
<4 m CE90
CURRENTLY IMAGING IN ORBIT
48. We have common frameworks to accelerate prototyping
We need a common data set to evaluate performance
49. Rio Olympics – Origins of SpaceNet
• Governments around the world were
preparing for the upcoming Olympics in
Rio De Janeiro
• Thorough understanding of the
security and safety of the athletes is
front of mind
• Knowledge of high risk areas, traffic
patterns, line-of-sight and overall
structure distribution is critical to ensure
a secure games
50.
51. - 50cm WV-2 image w/8 spectral bands
- Each image 200m2
- Covering Rio de Janeiro, Brazil
- 7000 Geotiff images
Available for your computing pleasure on
52. 1) Satellite imagery analytics will be way more common in your industry
2) Computer vision tasks are going to change the way we see the world
3) We are going to learn a lot
What You Can Expect
53. What You Can Do
ObservedEarth app observedearth.com
Australian Geoscience Data Cube: www.datacube.org.au
DigitalGlobe: aws.amazon.com/public-data-sets/spacenet/
developer.digitalglobe.com
Earth on AWS : aws.amazon.com/earth
AWS Open Data & Partners:
aws.amazon.com/government-education/open-data