Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
© 2015 IBM Corporation
Context Computing
Strata + Hadoop World 2015
Jeff Jonas, IBM Fellow
Chief Scientist, Context Comput...
© 2015 IBM Corporation
Jeff Jonas
IBM Fellow
Chief Scientist, Context Computing
 Founded Systems Research & Development (...
© 2015 IBM Corporation
No Context
Newsletter
Subscriber
© 2015 IBM Corporation
Context
“Better understanding something by taking into
account the things around it.”
© 2015 IBM Corporation
I ducked as the bat flew my way.
Another exciting baseball game.
© 2015 IBM Corporation
In Context
Social
Media
Influencer
Newsletter
Subscriber
Loyalty
Club Member
High Value
Customer
Jo...
© 2015 IBM Corporation
Context Accumulating
Context
Accumulation
Contextualized
Observations
Observation
(Any kind of data...
© 2015 IBM Corporation
Context Informs Decisioning
Context
Accumulation
Contextualized
Observations
Observation
In Context...
© 2015 IBM Corporation
The Puzzle Metaphor
 Imagine an ever-growing pile of puzzle pieces of varying sizes, shapes, color...
© 2015 IBM Corporation
Puzzling Images: Courtesy Ravensburger © 2011
270 pieces
90%
200 pieces
66%
150 pieces
50%
6 pieces...
© 2015 IBM Corporation
© 2015 IBM Corporation
© 2015 IBM Corporation
First Discovery
© 2015 IBM Corporation
More Data Finds Data
© 2015 IBM Corporation
Duplicates in Front Of Your Eyes
© 2015 IBM Corporation
First Duplicate Found Here
© 2015 IBM Corporation
© 2015 IBM Corporation
Incremental Context – Incremental Discovery
6:40pm START
22min “Hey, this one is a duplicate!”
35mi...
© 2015 IBM Corporation
150 pieces
50%
© 2015 IBM Corporation
Incremental Context – Incremental Discovery
47min “We should take the sky and grass off the table.”...
© 2015 IBM Corporation
© 2015 IBM Corporation
How Context Accumulates
 With each new observation one asserts: 1) unrelated; 2) related; or 3) co...
© 2015 IBM Corporation
Big Data [in context]. New Physics.
More data: better the predictions
– Lower false positives
– Lo...
© 2015 IBM Corporation
Big Data
Pile of ______ Information In Context
© 2015 IBM Corporation
One Essential Form of Context: “Entity Resolution”
 Is it 5 people each with 1 account or is it 1 ...
© 2015 IBM Corporation
Who is Fang Wong?
Fang Wong
Top 100 Customer
F A Wong
Seattle, DOB: 6/12/82
Former Customer
@FangWo...
© 2015 IBM Corporation
Resolving the Fang Wong
Fang Wong
Top 100 Customer
F A Wong
Seattle, DOB: 6/12/82
Former Customer
@...
© 2015 IBM Corporation
Resolving the Fang Wong
Fang Wong
Top 100 Customer
2.5M Followers
Newsletter Subscriber
© 2015 IBM Corporation
Graphing the (resolved) Fang Wong
Bill Smith
Member of the Board
Employee
Customer
Customer
Fraudst...
© 2015 IBM Corporation
Contextualizing Sandy Maden
Bill Smith
Member of the Board
Sandy Maden
New Account
Employee
Lives W...
© 2015 IBM Corporation
“Entities”
Bill Smith
Member of the Board
Lives With
Co-signer
Sandy Maden
New AccountFormer
Custom...
© 2015 IBM Corporation
Asteroid Hunting
© 2015 IBM Corporation
Single Detection
Image courtesy of: Eva Lilly, Institute of Astronomy, University of Hawaii
© 2015 IBM Corporation
From Orphans to Orbits
Single
Detections
(trash)
Tracklette
Track
Orbit
Forecasting
Named entity: S...
© 2015 IBM Corporation
http://www.space.com/7854-slam-asteroids-suspected-space-collision.html
© 2015 IBM Corporation
"We have directly
observed a collision
between asteroids for
the first time, instead of
having to i...
© 2015 IBM Corporation
Geospatial Context via “Space Time Boxes”
© 2015 IBM Corporation
Detecting Colocation
TIME
1 day
1 hour
Determine
encounter
distance
and time
Space Time Boxes
© 2015 IBM Corporation
Computing 600k Asteroid Interactions over 25 Years
4-5 orders of magnitude improvement
Initial Anal...
© 2015 IBM Corporation
Asteroid vs. Asteroid Encounters
Encounter Distance Asteroid 1 Size Asteroid 2 Size
May 1, 2032 299...
© 2015 IBM Corporation
June 12th, 2015
Hi Jeff & the gang,
I have great news! On Tuesday I happened to observe a close enc...
© 2015 IBM Corporation
Image courtesy of: Eva Lilly, Institute of Astronomy, University of Hawaii
© 2015 IBM Corporation
[Theatrical Pause]
© 2015 IBM Corporation
Action
Red Analytics
Green Analytics
Blue Analytics
Observation
Space
Old School: Isolated Analytics
© 2015 IBM Corporation
Observation
Space
ActionInformation
In Context
Next: General Purpose Context Computing
Data Finds D...
© 2015 IBM Corporation
Observation
Space
ActionInformation
In Context
Data Finds Data Relevance Finds You
Context Computin...
© 2015 IBM Corporation
Making Data Work: Recommendations
 Widen the observation space
 Accumulate context to improve und...
© 2015 IBM Corporation
More
 Blog: www.jeffjonas.typepad.com
 Email: JeffJonas@us.ibm.com
 Next: San Francisco, Nov 10-...
© 2015 IBM Corporation
Context Computing
Strata + Hadoop World 2015
Jeff Jonas, IBM Fellow
Chief Scientist, Context Comput...
Prochain SlideShare
Chargement dans…5
×

Strata hadoop world 2015 context computing - jonas keynote - final

0 vue

Publié le

Context Computing: My keynote at Strata & Hadoop World 2015.
http://strataconf.com/big-data-conference-ny-2015/

Publié dans : Données & analyses
  • DEAR FRIEND, I AM MR. MORRIS COULIBALY. I HAVE A LATE CLIENT WHO LEFT THE SUM OF $11. 5 MILLION DOLLARS IN OUR BANK I AM HIS PERSONAL ACCOUNT OFFICER I CONTACTED YOU BECAUSE YOU HAVE THE SAME LAST NAME OR SURNAME WITH THE DECEASED CLIENT AND I CAN PRESENT YOU AS THE BENEFICIARY AND NEXT OF KIN TO THE FUND SINCE YOU BEAR THE SAME SURNAME WITH MY LATE CLIENT. THE FUND WILL BE SHARE AMONG TWO OF US 50% EACH . I WAIT TO HEAR FROM YOU SO THAT I WILL GIVE YOU MORE DETAILS ON HOW THE FUND WILL BE RELEASE AND TRANSFER INTO YOUR BANK ACCOUNT . PLEASE CONTACT ME BACK IF YOU ARE INTERESTED FOR MORE DETAILS BEST REGARDS, . MR. MORRIS COULIBALY
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Strata hadoop world 2015 context computing - jonas keynote - final

  1. 1. © 2015 IBM Corporation Context Computing Strata + Hadoop World 2015 Jeff Jonas, IBM Fellow Chief Scientist, Context Computing http://www.twitter.com/jeffjonas www.jeffjonas.typepad.com
  2. 2. © 2015 IBM Corporation Jeff Jonas IBM Fellow Chief Scientist, Context Computing  Founded Systems Research & Development (SRD) in 1985  Architected, designed, developed roughly 100 systems over the last three decades – Financial services – Defense, intelligence – Manufacturing – Humanitarian efforts  Acquired by IBM in 2005  Currently focused on Context Computing, Sensemaking and Privacy by Design
  3. 3. © 2015 IBM Corporation No Context Newsletter Subscriber
  4. 4. © 2015 IBM Corporation Context “Better understanding something by taking into account the things around it.”
  5. 5. © 2015 IBM Corporation I ducked as the bat flew my way. Another exciting baseball game.
  6. 6. © 2015 IBM Corporation In Context Social Media Influencer Newsletter Subscriber Loyalty Club Member High Value Customer Job Applicant Watch Listed Party
  7. 7. © 2015 IBM Corporation Context Accumulating Context Accumulation Contextualized Observations Observation (Any kind of data from any kind of sensor)
  8. 8. © 2015 IBM Corporation Context Informs Decisioning Context Accumulation Contextualized Observations Observation In Context Decisioning Act Data Finds Data Relevance Finds You Observation (Any kind of data from any kind of sensor)
  9. 9. © 2015 IBM Corporation The Puzzle Metaphor  Imagine an ever-growing pile of puzzle pieces of varying sizes, shapes, colors  What it represents is unknown – there is no picture on hand  Is it one puzzle, 15 puzzles, or 1,500 different puzzles?  Some pieces are duplicates, missing, incomplete or have errors  Some pieces may even be professionally fabricated lies  Until you take the pieces to the table, it is nearly impossible to assess the scene
  10. 10. © 2015 IBM Corporation Puzzling Images: Courtesy Ravensburger © 2011 270 pieces 90% 200 pieces 66% 150 pieces 50% 6 pieces 2% 30 pieces 10% (duplicates)
  11. 11. © 2015 IBM Corporation
  12. 12. © 2015 IBM Corporation
  13. 13. © 2015 IBM Corporation First Discovery
  14. 14. © 2015 IBM Corporation More Data Finds Data
  15. 15. © 2015 IBM Corporation Duplicates in Front Of Your Eyes
  16. 16. © 2015 IBM Corporation First Duplicate Found Here
  17. 17. © 2015 IBM Corporation
  18. 18. © 2015 IBM Corporation Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” 35min “I think some pieces are missing.” 37min “Looks like a bunch of hillbillies on a porch.” 44min “Hillbillies, playing guitars, sitting on a porch, near a barber sign and a banjo!”
  19. 19. © 2015 IBM Corporation 150 pieces 50%
  20. 20. © 2015 IBM Corporation Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” 2hr10m “Wait, there are three … no, four puzzles.” 2hr18m “I think you threw in a few random pieces.”
  21. 21. © 2015 IBM Corporation
  22. 22. © 2015 IBM Corporation How Context Accumulates  With each new observation one asserts: 1) unrelated; 2) related; or 3) connected  Must favor the false negative  New observations sometimes reverse earlier assertions  Some observations produce novel discovery  The emerging picture helps focus collection interests
  23. 23. © 2015 IBM Corporation Big Data [in context]. New Physics. More data: better the predictions – Lower false positives – Lower false negatives More data: bad data good – Suddenly glad your data is not perfect More data: less compute
  24. 24. © 2015 IBM Corporation Big Data Pile of ______ Information In Context
  25. 25. © 2015 IBM Corporation One Essential Form of Context: “Entity Resolution”  Is it 5 people each with 1 account or is it 1 person with 5 accounts?  Is it 20 cases of SARS in 20 cities or one case reported 20 times?  If one cannot count, one cannot estimate vector or velocity (direction, speed).  Without vector and velocity prediction is nearly impossible.
  26. 26. © 2015 IBM Corporation Who is Fang Wong? Fang Wong Top 100 Customer F A Wong Seattle, DOB: 6/12/82 Former Customer @FangWong 2.5M Followers FangWong@Email.com Newsletter Subscriber Fang Wong FangWong@Email.com Marketing Department’s Prospect List
  27. 27. © 2015 IBM Corporation Resolving the Fang Wong Fang Wong Top 100 Customer F A Wong Seattle, DOB: 6/12/82 Former Customer @FangWong 2.5M Followers FangWong@Email.com Newsletter Subscriber Fang Wong FangWong@Email.com Marketing Department’s Prospect List
  28. 28. © 2015 IBM Corporation Resolving the Fang Wong Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber
  29. 29. © 2015 IBM Corporation Graphing the (resolved) Fang Wong Bill Smith Member of the Board Employee Customer Customer Fraudster Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber
  30. 30. © 2015 IBM Corporation Contextualizing Sandy Maden Bill Smith Member of the Board Sandy Maden New Account Employee Lives With Co-signer Former Customer Customer Customer Customer Fraudster Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber
  31. 31. © 2015 IBM Corporation “Entities” Bill Smith Member of the Board Lives With Co-signer Sandy Maden New AccountFormer Customer Employee Customer Customer Customer Fraudster Fang Wong Top 100 Customer 2.5M Followers Newsletter Subscriber Company Boat Plane Asteroid Car
  32. 32. © 2015 IBM Corporation Asteroid Hunting
  33. 33. © 2015 IBM Corporation Single Detection Image courtesy of: Eva Lilly, Institute of Astronomy, University of Hawaii
  34. 34. © 2015 IBM Corporation From Orphans to Orbits Single Detections (trash) Tracklette Track Orbit Forecasting Named entity: S100ZUtza Single Detection (orphan) Anticipation
  35. 35. © 2015 IBM Corporation http://www.space.com/7854-slam-asteroids-suspected-space-collision.html
  36. 36. © 2015 IBM Corporation "We have directly observed a collision between asteroids for the first time, instead of having to infer that they happened from million-year-old remains." Colin Snodgrass Planetary Scientist Max Planck Institute for Solar System Research
  37. 37. © 2015 IBM Corporation Geospatial Context via “Space Time Boxes”
  38. 38. © 2015 IBM Corporation Detecting Colocation TIME 1 day 1 hour Determine encounter distance and time Space Time Boxes
  39. 39. © 2015 IBM Corporation Computing 600k Asteroid Interactions over 25 Years 4-5 orders of magnitude improvement Initial Analysis Adding 1 New Trajectory Space-Time Box Method 2,880 CPU hours 15 CPU minutes N-body Simulation Method 10,000,000 CPU hours 4,000 CPU hours
  40. 40. © 2015 IBM Corporation Asteroid vs. Asteroid Encounters Encounter Distance Asteroid 1 Size Asteroid 2 Size May 1, 2032 299km 00A9170 2-4km 0008758 4-9km Nov 24, 2016 449km 00P5634 1-2km 0055711 2-5km Jan 11, 2018 449km K08E88J 530-1200m 00N0062 2-4km
  41. 41. © 2015 IBM Corporation June 12th, 2015 Hi Jeff & the gang, I have great news! On Tuesday I happened to observe a close encounter you guys predicted - one 1 km and the other one 2 km in diameter! To my knowledge this is the first case ever of direct observation of a close encounter in the small main belt asteroids. The closest point of encounter unfortunately happened during bright daylight in Hawaii, so I missed it … Cheers! Eva -
  42. 42. © 2015 IBM Corporation Image courtesy of: Eva Lilly, Institute of Astronomy, University of Hawaii
  43. 43. © 2015 IBM Corporation [Theatrical Pause]
  44. 44. © 2015 IBM Corporation Action Red Analytics Green Analytics Blue Analytics Observation Space Old School: Isolated Analytics
  45. 45. © 2015 IBM Corporation Observation Space ActionInformation In Context Next: General Purpose Context Computing Data Finds Data Relevance Finds You Context Computing
  46. 46. © 2015 IBM Corporation Observation Space ActionInformation In Context Data Finds Data Relevance Finds You Context Computing Helping Focusing Human Attention General Purpose • Marketing • Customer service • Fraud detection • Asteroid hunting Simultaneously!
  47. 47. © 2015 IBM Corporation Making Data Work: Recommendations  Widen the observation space  Accumulate context to improve understanding  Deliver significantly higher quality outcomes … everywhere – Life sciences – Financial services – Public safety  Leverage Hadoop/Spark to accelerate innovation
  48. 48. © 2015 IBM Corporation More  Blog: www.jeffjonas.typepad.com  Email: JeffJonas@us.ibm.com  Next: San Francisco, Nov 10-12, @Datapalooza
  49. 49. © 2015 IBM Corporation Context Computing Strata + Hadoop World 2015 Jeff Jonas, IBM Fellow Chief Scientist, Context Computing http://www.twitter.com/jeffjonas www.jeffjonas.typepad.com

×