Presented by Ganes Kesari at the Intelligent Cloud Conference, Copenhagen on 9-Apr-2019 (http://intelligentcloud.dk)

1. Session
Ganes Kesari
Gramener
How Deep Learning is Saving the Planet
Session

2. 2
INTRODUCTION
Co-founder & Head
of Analytics
“Simplify Data
Science for all”
100+ Clients
Insights as Stories
Help apply & adopt
Analytics
Our data science platform,
Gramex is now open-
sourced!

3. 3
OUR STORY BEGINS 50,000 YEARS AGO…
What happened to them?
https://wattsupwiththat.com/2017/01/20/humans-not-climate-change-wiped-out-australian-megafauna/

4. 4
Humans are ecological serial killers…“
http://www.wrbh.org/wp-content/uploads/2018/05/Spaiens-book-cover.jpg
…even with stone-age tools, our ancestors wiped
out half the planet’s large terrestrial mammals.
- Yuval Noah Harari

5. 5
CAN WE SAVE OUR BIODIVERSITY?

6. 6
SPOTTING, IDENTIFYING AND COUNTING ANIMALS TO SAVE THEM
Gramener has partnered with Microsoft AI for Earth
https://www.microsoft.com/en-us/ai/ai-for-earth

7. 7
MACHINE LEARNING 101
New Input
Desired
Outcome
Machine learning
how to do the job
Known Input
Known
Outcome
“Programs that solve
the problem”
“Programs that learn
to solve the problem”
vs

8. 8
WHY DEEP LEARNING?
8
Input Output
Identify features
to teach model
Traditional Machine Learning
Deep Learning
Person
Name
Input Output
Model automatically identifies
features to learn
Person
Name
https://www.cs.toronto.edu/~ranzato/publications/taigman_cvpr14.pdf

9. 9
NOT VERY DIFFERENT FROM HOW WE LEARN
Training ..Versatile Detection!

10. 10
WHERE HAVE THE SALMON GONE?
Source: Giphy (https://media.giphy.com/media/QM5GJO6J8lDfa/giphy.mp4)

11. 11
CAN YOU IDENTIFY THESE SALMON SPECIES?
Sockeye Steelhead

12. 12
MONITORING SALMON MOVEMENT

13. 13
BUILDING THE MODEL – FASTER RCNN
https://tryolabs.com/blog/2018/01/18/faster-r-cnn-down-the-rabbit-hole-of-modern-object-detection/
Region Proposal Network Region of Interest Pooling Regional CNN

14. 14
CLASSIFYING THE 12 SALMON SPECIES

15. 15
THE MODEL IN ACTION
Video

16. 16
A VISUAL NARRATIVE OF THE ENGAGEMENT
Microsoft published case study on this project at: https://partner.microsoft.com/en-us/case-studies/gramener/

17. 17
SPOTTING ELEPHANTS IN THE WILD
Source: Giphy (https://media.giphy.com/media/1AHZzdXVTYDtJVTO5a/giphy.mp4)

18. 18
CAN YOU SPOT THE ELEPHANTS?
https://www.savetheelephants.org/project/tsavo-aerial-defence/

19. 19
THE DATA & CHALLENGES
Annotate

20. 20
MODEL BUILDING
Region Proposal Network Region of Interest Pooling Regional CNN
FASTER RCNN
SSD MULTIBOX

21. 21
MODEL RESULTS

22. 22
CAN WE CLASSIFY ALL SPECIES ON EARTH?
Source: Giphy (https://media.giphy.com/media/CFk1wEH7Cke0E/giphy.mp4)

23. 23
INATURALIST: FOSTERING CITIZEN SCIENCE
https://play.google.com/store/apps/details?id=org.inaturalist.android&hl=en

24. 24
BUILDING THE MODEL
Inception v4

25. 25
LIVE API DEMO

26. 26
LET’S SAVE THE PENGUINS
Penguin populations are at risk
in Antarctica and researchers
need help to detect how it’s
reducing over the years
Source: Giphy (https://giphy.com/gifs/push-bwLowbhUWm2lO)

27. 27
TRACKING THE PENGUINS
https://www.zooniverse.org/projects/penguintom79/penguin-watch/about/research
~100 cameras in 16 locations, and hourly
images over years
Crowd-sourced annotations to
identify the penguins

28. 28
CROWD-SOURCED ANNOTATIONS
https://www.zooniverse.org/projects/penguintom79/penguin-watch/classify

29. 29
APPROACHES TO COUNTING CROWDS
Occlusion
Density Difference
Perspective Distortion
Camera angle

30. 30
COUNTING USING DENSITY-BASED ESTIMATIONS
Preserve spatial information
Localize count
Handle scale variations
No longer looking for a head!

31. 31
APPROACH IN BRIEF
https://arxiv.org/pdf/1707.09605.pdf
Input an image Density Map Estimate the countSplit into 9 patches

32. 32
CHALLENGES WITH THE DATA
Cleaned dataset:
• Training: 18k
• Validation: 3k
• Test: 9k
Hurdles
• Camera angles,
• Occlusion,
• Perspective distortion,
• Density difference,
• Weather conditions
Work done by Gramener in partnership with Microsoft AI for Earth

33. 33
MODEL ARCHITECTURE
• High-level prior to classify image into buckets
• Density estimation to create the density map
• NC6 v3 virtual machine with V100 GPU card
• Trained for 200 epochs, MAE for the model : ~10.5
https://arxiv.org/pdf/1707.09605.pdf Work done by Gramener in partnership with Microsoft AI for Earth

34. 34
WALKTHROUGH

35. 35
DEEP LEARNING TAKEAWAYS: WHEN THE RUBBER HITS THE ROAD
• Acquire & clean data
• Label your own data
• Look out for practical data challenges
• Don’t stop at counts - go for actionability
• Build it into the user’s natural workflow
• Sensitize users on low accuracy
• Plan for model refresh

36. 36
The first step towards change is awareness. The
second is acceptance.
– Nathaniel Branden
“
WildMe projects - https://www.wildbook.org/ https://www.flukebook.org/
THE WAY FORWARD..

37. 37
Session
@kesaritweetsgramener.com @kesari
Presentation deck with references at
gkesari.com/intelligentcloud

38. Event
partners
Expo
partners
Expo light
partners