Talk given at the Sensor and Sensor Network Technologies in Environmental Monitoring workshop. May 8-9th, 2013 Canberra

1. Gigapixel resolution imaging for
near-remote sensing and
phenomics
7.3 billion pixel image of the National Arboretum, Canberra, ACT, Australia
Assembled from 900, 18MP images. (http://gigapan.com/gigapans/120215)
Magpies (1.3km) ANU resaerch forest (500m)
Dr. Tim Brown, Borevitz Lab, Australia National University

2. Paradigms
How do we do science?
– Use limited data to build models of how the world works
• The Old: data limited
– Sit under a tree and record what you see
– Sample what you can afford to for a limited period of
time
– Usually temp/hg being recorded; if you’re lucky there’s
satellite data
– Very low res version of reality
• The future: software limited (and paradigm limited)
– Long time series
– High spatial and temporal resolution
– Flux towers, mesh networks, gigapixel cameras, UAV’s,
3D models of ecosystems

3. We are building the tools to enable
“NextGen” ecology
• “Total ecosystem awareness”
– What hardware do we need to do this?
– What novel data streams exist that we can
use?
– The hardware questions will be solved, but…

4. Imagine if we could watch every plant in every
research sites from our desks?
– Map out population genetics, biotic/abiotic data on the
landscape
• Light sequence every plant in the landscape (~$10/plant)
– Slide back in time and watch any interaction for as long as
there have been sensors
– Students start new research projects beginning with all the
data previously collected at a site
– View time-series data in situ, on site with a tablet
• Tons of opportunities here
The technology is (almost) here
We need to dream big!

5. • A lot is going at every scale from the individual plant to a
whole ecosystem.
• Depending on what questions you want to answer some
of this detail is may really matter
• See the timelapse here:
www.youtube.com/watch?v=ymYQCrmDN8Y&hd=1

6. What technologies enable this new way
of monitoring ecosystems?
• Mesh networks
• Internet
– Need standards
• Gigapixel imaging
• UAVs
• Low cost land/UAV-based LIDAR
• Smartphone science
• Automated processing & QC/QA
Need better software at all levels

7. Collaboration between
Borevitz Lab (U. Chicago, now ANU) and TimeScience (my company)
The Challenge:
• Build a solar powered, weatherproof gigapixel camera that can record
daily phenology from every plant in a field area.
Gigavision: gigapixel timelapse camera
From Gigapan to gigapixel timelapse

8. (Single 15MP image)
Area: ~7ha
Area: ~1m2
The Gigapan and
Gigavision systems
allow you to capture
hundreds or
thousands of
zoomed-in images in
a panorama.
Images are then
“Stitched” into a
seamless panorama.
The super-high
resolution of the final
panorama lets you
monitor huge
landscape areas in
great detail.
Gigapixel Imaging – How it works

9. • ~1.5 billion pixels / panorama (could be more)
• Avg. resolution of ~1 pixel / cm over 7 hectares
– (~600 million times the pixel resolution of MODIS)
• Open-source - Built with off-the-shelf components
– Gigapixel imaging isn’t that hard. Really the hard part is Software (again)
• Cellular (3G) or 802.11g wireless access (160MP “thumbnails”)
• Automated capture up to 1 image / hr
• Solar powered (<15w power consumption)
• $30K - $40K (could be more like $10-15,000)
• “Light-phenotyping” of >500 plants for ~$60/plant
Gigavision Camera – Specifications
For full specs, see Brown et al. 2012
(Google: “gigavision chapter”)

10. Gigavision Camera Location:
“Big Blowout East”
Indiana Dunes State Park, IN, USA

12. Camera Field of View (FOV)

13. Camera Field of View (FOV)
Actual camera view

14. Dataset statistics
• Time period recorded: Oct 2009 – Oct 2011
– 2 N. Hemisphere growing seasons (April – Oct)
• 1 - 4 panoramas / day (~154 15MP images/panorama)
• Initial image dataset: ~184,000 individual jpg images
• Processed data = ~70 million 200x200px image tiles
• 6TB of space
• 417 usable noon panoramas

15. Gigavision camera coverage, 2009 - 2011
Growing seasons
Colors denote a successful panorama captured at the indicated time of day.
Note missing data at key points in the spring (N. Hemisphere spring).
When each season is a data point it is important to have support staff
available at point when the hardware must not fail.

16. Image Visualization and Data Collection
Browse the timelapse online: http://bit.ly/GVDemo2012-1
Data Collection demo: http://bit.ly/GVDemoMovie2012

17. Data Collection demo movie:
http://bit.ly/GVDemoMovie2012

18. • 513 individual plants identified
• 8 prominent species (non grasses)
• Species:
– Hoary Puccoon = 344
– Unidentified (yet) = 52
– Cottonwood = 47
– Black Oak = 36
– Sand Cherry = 18
– Juniper = 9
– Wormwood = 3
– Pitcher’s Thistle (Endangered) = 2
– Marsh Marigold = 2
Indiana Dunes Gigavision Camera Data Summary

19. Gigapan – Low Cost Gigapixel Imaging
Non-timelapse Gigapixel robotic camera heads by
GigaPan) cost $300-$900 and work with any camera
• Great for:
• Repeat photography and monitoring
• Site Documentation and detection
• Visualization; grant proposals; impressing
funders!
• More examples of gigapans here:
http://gigapan.com/profiles/TimeScience
• Camera hardware: http://www.gigapan.com/

20. Alta Ski Area Bark Beetle Project
Collaboration w/ Maura Olivos at the Alta Environmental Center
Goal:
• Improve early detection of bark beetle outbreaks within Alta
Ski Area’s (Alta, UT, USA) management area
Solution
• Augment traditional aerial survey data with annual gigapixel
photo surveys to enhance detection success.
• Gigapixel imaging provides a new survey tool that allows
Alta staff to examine the health of almost every tree in a
810ha area via a web interface.

21. Path data collected with EveryTrail smartphone app (http://www.everytrail.com/ )
Initial survey path for potential panorama locations
View the panoramas from the initial survey: http://gigapan.org/galleries/6787/gigapans

22. Greeley
Approximate coverage of the four Gigapan survey images.
[Note that an additional image taken from the Greeley area (green arrow) would cover most of the missing area
on the east side of Alta. This area is currently not at high risk for beetle outbreaks and so was not surveyed.]

23. (1) Collins Weather
(2) Baldy Shoulder
(3) Road Shot
(4) Grizzly
View all survey panoramas online here: http://gigapan.org/galleries/5582/gigapans

24. (1) Collins Weather
(2) Baldy Shoulder
(3) Road Shot
(4) Grizzly
Example of beetle-killed tree detected in image.

25. Project summary statistics
Site Name Approximate Area
(Hectares)
Image Resolution
(Gigapixels)
Average Pixel Resolution
(Pixels per square inch)
Collins Weather Station 282 4.07 0.93
Baldy Shoulder 312 8.84 1.83
Road Shot 85 3.26 2.46
Grizzly Gulch 76 3.65 3.1
TOTALS 810 ha Total: 23.5 billion px
Avg: 4.7 billion px
Avg: ~2 px/in2
(Area Estimates: http://www.earthpoint.us/Shapes.aspx)
Online Data collection portal
http://bit.ly/Rvn4p3

26. Australian time-series monitoring
• Canberra, Telstra Tower
• National Arboretum
Gigapixel imaging lets us monitor seasonal change for huge areas

27. 20 gigapixel image of Canberra, Australia from the Black Mountain Telstra Tower
Zoom in to the National Arboretum

28. Midsummer
Zoom in to the Each forest at the Arboretum

29. Low cost sequencing let’s us genotype every individual tree and identify genetic loci that
correlate with observed phenotypic differences between trees.
We can do this for all trees at the arboretum within view of the camera.
Fall Color change shows differing rates of fall senescence in trees
Late fall

30. Gigavision cameras with mesh sensor networks
Nat’l Arboretum & ANU Coastal Campus (2013 LIEF proposal)

31. UAVs
• Increasingly cheap and easy to use
– DIY version is ~$600 with full GPS and
autopilot
• Commercial solutions are $10K - $100K
• Commercial software (e.g. Pix4D)
– $10-15K
– 100mm resolution DEMs + image layers
– Some pay per use and cloud based solutions
are emerging
• Regulatory framework is a challenge

33. • Test aerial survey using a 40MP
Nokia cell phone and a low cost
quadcopter.
• Images stitched in Autopano Giga
software yield a 1 billion pixel image.
• Autopilot enables easy repeat
surveys.
• See full panorama here:
– http://gigapan.com/gigapans/127099

34. Smartphones
Image: 40 consumer gadgets have converged into one device.
: http://www.wired.com/magazine/2013/04/convergence/

35. We shouldn’t underestimate the utility of
smartphones
• There are currently more then 1 billion
smartphones in use globally
• Phone tech:
– 4G networks have the same bandwidth as MODIS
– More processing power than most satellites
– Can sense natively:
• Position (w/in 3-4m), Temp, Light, magenetic field, orientation, proximity,
sound levels
– Have internet; OS, can ftp images, data, etc
– Can take up 40mp images (Nokia 808)

36. Citizen Science and Crowdsourced Science
There are many sources of data coming online
that can provide useful time-series information
about the world.
Example:
• Geo-tagged photos
– Facebook:
• 300 million images uploaded / day w/240 billion total
– This is up from 83 million 2 years ago
• Most (?) are from mobile and geolocated
– US National Parks – 280 million visitors / yr
• How many of these people visit the same spot, take the same
geolocated picture with their smartphone and then put it online?
• How do we begin to access and use data sets like this?

37. • Map of pixels of how much of the world
has been captured by geo-referenced
image on flickr (not including google street
view of course).
© Eric Fischer (http://www.flickr.com/photos/walkingsf/ )

38. Geotagged Chicago
© Eric Fischer (http://www.flickr.com/photos/walkingsf/ )

39. Photosynth
Microsoft and U. Washington PhotoTourism and Photosynth projects
• http://photosynth.net/ (free online tools for object mapping with images)
• http://grail.cs.washington.edu/projects/rome/
• http://phototour.cs.washington.edu/findingpaths/
Photosynth Project – Building 3D maps of the world from online images

41. For more information
Primary project contact (all projects)
Tim Brown ANU, Time-Science (tim@time-science.com)
Ph: +1 801-554-9296
Skype: TimeScience
Additional project participants
Camera Systems and Data Visualization
Christopher Zimmermann
www.time-science.comwww.time-science.com
Gigavision Camera Project
Justin Borevitz
Nina Noah, Whitney Panneton
University of Chicago
Data: http://gigavision.org
Purchase: http://gigavision.net
Alta Bark Beetle Project
Tim Brown
Maura Olivos
Alta Ski Area / Alta Environmental Center
http://www.altaence.com/