SlideShare a Scribd company logo
1 of 44
Download to read offline
DATA MINING AND MACHINE LEARNING
                                                                 IN A NUTSHELL



           COLLECTIVE INTELLIGENCE
                                                                        PART I

                                                    Mohammad-Ali Abbasi
                                                          http://www.public.asu.edu/~mabbasi2/

                                     SCHOOL OF COMPUTING, INFORMATICS, AND DECISION SYSTEMS ENGINEERING
                                                         ARIZONA STATE UNIVERSITY

              Arizona State University
                                                                http://dmml.asu.edu/
Data Mining and Machine Learning Lab
                                         Data Mining and Machine Learning- in a nutshell              Collective Intelligence   1
About Collective Intelligence

  • Definition of collective intelligence
         – Examples happening around us
  • What constitutes collective intelligence
         – Groups, number of members, variety, etc.
  • How can one improve collective intelligence
         – What are necessary conditions to achieve CI
         – A case in data mining and machine learning?
  • What can one do with collective intelligence in
    the age of social media
         – Opportunities for Data Mining
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   2
Definitions for Collective intelligence

  • Wikipedia
         – Collective intelligence is a shared or group intelligence that
           emerges from the collaboration and competition of many
           individuals
  • MIT Center for CI
         – Groups of individuals doing things collectively that seem
           intelligent
  • Toby Segaran in Programming CI
         – Combining the behavior, preferences, or ideas of a group
           of people to create novel insights
  • Unknown
         – Collective intelligence is any intelligence that arises from -
           or is a capacity or characteristic of - groups and other
           collective living systems
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   3
Examples of collective intelligence - Wikipedia

  • Wikipedia
  • Thousands of contributors from across the
    world have collectively created the world’s
    largest encyclopedia
  • with almost no centralized control




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   4
Examples of collective intelligence - PageRank

  • PageRank Algorithm used by Google




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   5
Examples of collective intelligence - CAPTCHA

  • CAPTCHA
         – Completely Automated Public Turing test to tell Computers and Humans Apart
         – A reverse Turing test (machine to human instead of human to machine)

  • A service that helps to digitize books,
    newspapers and old time radio shows
         – About 200 million CAPTCHAs are solved by
           humans around the world every day
         – More than 150,000 hours of work each day




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   6
Vark.com

  1. Send a question
  2. Aardvark finds the perfect person to answer
  3. Get their response




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   7
Kasparov vs. the World

  • Kasparov v. the World was a chess match held
    in 1999, when world champion Gary Kasparov
    played against “the World,” with the World’s
    moves determined by majority vote over the
    Internet of anyone who wanted to participate.


  Kasparov eventually won, but he said it
    was the hardest game he ever played


                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   8
Examples of collective intelligence - Threadless

  • Threadless.com
  • In Threadless, anyone who
    wants to can design a T-
    shirt, submit that design
    to a weekly contest, and
    vote for their favorite
    designs
  • the company harnesses
    the collective intelligence
    of a community of over
    500,000 people to design
    and select T-shirts

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   9
Examples of collective intelligence –Google Image
Labeler
  • It is a feature, in the form of a game,
    of Google Image Search that allows the user to
    label random images to help improve the
    quality of Google's image search results




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   10
Examples of collective intelligence – Ant Societies

  • Ant societies exhibit more intelligence than
    any other animal except for humans, if we
    measure intelligence in terms of technology.
    Ant societies are able to do agriculture, in fact,
    in several different forms of agriculture. Some
    ant societies keep livestock of various forms,
    for example, some ants keep and care for
    aphids for "milking”; Leaf cutters care for
    fungi and carry leaves to feed the fungi.


                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   11
Examples of collective intelligence - Games

  • Games such as WorldCraft, The Sims, Halo or
    Second Life are designed to be more non-
    linear and depend on collective intelligence
    for expansion.
  • This way of sharing is gradually evolving and
    influencing the mindset of the current and
    future generations.




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   12
Principals of Collective Intelligence

  • Collective intelligence is of mass collaboration.
    In order for collective intelligence to emerge,
    four principles exist to promote creativity:
         – Openness
         – Peering
         – Sharing and
         – Acting globally




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   15
Openness

  • Traditionally, people and companies are naturally
    reluctant to share ideas and intellectual property
    because these resources provide the edge over
    competitors.
  • However, in time, openness is promoted when
    people and companies began to loosen hold over
    these resources as they reap more benefits in
    doing so.
  • Openness enables products to gain significant
    improvement and scrutiny through transparent
    collaboration.
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   16
Peering
  • A form of horizontal organization with the capacity to
    create information technology and physical products.
  • One example is the ‘opening up’ of the Linux program
    where users are free to modify and develop it provided that
    they made it available for others.
  • Participants in this form of collective intelligence may have
    different motivations for contributing, but the results
    achieved are for the improvement of a product or service.
  • “Peering succeeds because it leverages self-organization – a
    style of production that works more effectively than
    hierarchical management for certain tasks.”




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   17
Sharing

  • Research has shown that more and more
    companies have started to share some, while
    maintaining some degree of control over
    others, like potential and critical patent rights.
  • This is because companies have realized that
    by limiting all their intellectual property, they
    are shutting out all possible opportunities.
  • Sharing some has allowed them to expand
    their market and bring out products faster.

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   18
Acting Globally

  • The advancement in communication technology
    has prompted the rise of global companies, or e-
    Commerce that has allowed individuals to set up
    businesses at low to almost no overhead costs.
  • The influence of the Internet is widespread,
    therefore a globally integrated company would
    have no geographical boundaries but have global
    connections, allowing them to gain access to new
    markets, ideas and technology.
  • Therefore it is important for firms to get updated
    and remain globally competitive or they will face
    a declining rate of clients.

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   19
Types of Collective Intelligence




              Arizona State University
Data Mining and Machine Learning Lab
                                         Data Mining and Machine Learning- in a nutshell   Collective Intelligence   20
Elements of Collective Intelligence

  • Staffing
         – Who is performing the task?
  • Incentives
         – Why are they doing it?
  • Goal
         – What is being accomplished?
  • Structure, process
         – How is it being done?

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   21
Elements of Collective Intelligence
  • Who?
         – Hierarchy
         – Crowd
  • Why?
         – Money
         – Love
         – Glory
  • What?
         – Create
         – Decide
  • Who
         – Collection
         – Collaboration

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   22
Mapping the collective intelligence elements for
Wikipedia




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   23
Issues with Crowd Wisdom

  • Questions
         – Why can the crowd be smarter than any individual
           in the crowd?
         – Is it guaranteed? If not, what are the conditions
           under which the crowd can make best decisions?
         – How can one gauge the reliability of crowd
           wisdom? Is crowd wisdom valid, trustworthy, and
           verifiable?
         – How to find a crowd, its leader/influencer/average
           opinion?
         – How is each member influenced by others?
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   24
Collective Intelligence and Societies

  • The main base of all kinds of CI’s is society
  • CI in traditional societies
         – Families, companies, countries, and armies are all
           groups of individuals doing things collectively that,
           at least sometimes, seem intelligent
  • CI in Web based societies- Social Networking
    sites
         – Internet and specially Web 2.0 applications
           provide a platform for communications and
           building societies
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   25
Collective Intelligence
                                     and the Internet


                                  • Web 2.0
                                  • Social Computing

              Arizona State University
Data Mining and Machine Learning Lab
                                         Data Mining and Machine Learning- in a nutshell   Collective Intelligence   26
Web Impacts on CI

  • The ability of new media to easily store and
    retrieve information, predominantly through
    databases and the Internet, allows it to be
    shared without difficulty.
  • Thus, through interaction with new media,
    knowledge easily passes between sources
    resulting in another form of collective
    intelligence


                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   27
WEB 2.0 and Many Variants




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   28
Elements of WEB 2.0




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   29
Web 2.0: Evolution Towards a Read/Write Platform

            Web 1.0                                                                                      Web 2.0
                       (1993-2003)                                                                             (2003- beyond)

  Pretty much HTML pages viewed through a                                                      Web pages, plus a lot of other “content” shared
                 browser                                                                      over the web, with more interactivity; more like an
                                                                                                          application than a “page”
                       “Read”                                                Mode                   “Write” & Contribute


                       “Page”                                       Primary Unit of                       “Post / record”
                                                                        content

                      “static”                                               State                            “dynamic”

               Web browser                                        Viewed through…                Browsers, RSS Readers,
                                                                                                       anything

             “Client Server”                                          Architecture                       “Web Services”

                 Web Coders                                    Content Created by…                             Everyone

                      “geeks”                                          Domain of…                 “mass amatuerization”


                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell                        Collective Intelligence          30
CI in Social Media

  • Crowd members assign different weights to
    individual inputs on the basis of their
    relationship with the people who provided
    them and then make individual decisions
         – Blogosphere
         – Facebook
         – YouTube
         – Epinions.com
         – Amazon
         – eBay
         – Digg
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   31
Blogging is the Most Recognized Example of Web 2.0




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   32
Blogging is the Most Recognized Example of Web 2.0




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   33
Wikipedia is a Collaborative Dictionary Being Edited in Real-time by Anyone




                    Arizona State University
      Data Mining and Machine Learning Lab
                                               Data Mining and Machine Learning- in a nutshell   Collective Intelligence   34
Alive At ASU




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   35
WEB 2.0 Technologies

  • APIs
  • RSS (Really Simple Syndication)
         – Content Syndication
  • Web Services
         – Open Data
  • AJAX (Asynchronous Javascript and XML)
  • CSS (Cascading Style Sheets)
         – Content with Style

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   36
WEB 2.0, Summing Up
• Web 2.0 hard to define, but very far from just hype
       – Culmination of a number of Web trends
• Importance of Open Data
       – Allows communities to assemble unique tailored
         applications
• Importance of Users
       – Seek and create network effects
• Browser as Application Platform
       – Huge potential for new kinds of Web applications


               Arizona State University
 Data Mining and Machine Learning Lab
                                          Data Mining and Machine Learning- in a nutshell   Collective Intelligence   37
Programming
Collective Intelligence




               Arizona State University
 Data Mining and Machine Learning Lab
                                          Data Mining and Machine Learning- in a nutshell   Collective Intelligence   38
Crawl the web




              Arizona State University
Data Mining and Machine Learning Lab
                                         Data Mining and Machine Learning- in a nutshell   Collective Intelligence   39
Spiders (Robots/Bots/Crawlers)

  • Start with a comprehensive set of root URL’s
    from which to start the search.
  • Follow all links on these pages recursively to
    find additional pages.
  • Index all novel found pages in an inverted
    index as they are encountered.
  • May allow users to directly submit pages to be
    indexed (and crawled from).

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   40
Search Strategies



                                                      Breadth-first Search




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   41
Search Strategies (cont)



                                                         Depth-first Search




                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   42
Search Strategy Trade-Off’s

  • Breadth-first explores uniformly outward from
    the root page but requires memory of all
    nodes on the previous level (exponential in
    depth). Standard spidering method.
  • Depth-first requires memory of only depth
    times branching-factor (linear in depth) but
    gets “lost” pursuing a single thread.
  • Both strategies implementable using a queue
    of links (URL’s).

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   43
Spidering Algorithm

  • Initialize queue (Q) with initial set of known URL’s.
  • Until Q empty or page or time limit exhausted:
         – Pop URL, L, from front of Q.
         – If L is not to an HTML page (.gif, .jpeg, .ps, .pdf, .ppt…)
                     • continue loop.
         – If already visited L, continue loop.
         – Download page, P, for L.
         – If cannot download P (e.g. 404 error, robot excluded)
                     • continue loop.
         – Index P (e.g. add to inverted index or store cached copy).
         – Parse P to obtain list of new links N.
         – Append N to the end of Q.

                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   44
Keeping Spidered Pages Up to Date

  • Web is very dynamic: many new pages, updated
    pages, deleted pages, etc.
  • Periodically check spidered pages for updates and
    deletions:
         – Just look at header info (e.g. META tags on last update) to
           determine if page has changed, only reload entire page if
           needed.
  • Track how often each page is updated and
    preferentially return to pages which are historically
    more dynamic.
  • Preferentially update pages that are accessed more
    often to optimize freshness of more popular pages.
                 Arizona State University
   Data Mining and Machine Learning Lab
                                            Data Mining and Machine Learning- in a nutshell   Collective Intelligence   45
Mohammad-Ali Abbasi (Ali),
                                         Ali, is a Ph.D student at Data Mining
                                         and Machine Learning Lab, Arizona
                                         State University.
                                         His research interests include Data
                                         Mining, Machine Learning, Social
                                         Computing, and Social Media Behavior
                                         Analysis.

                                         http://www.public.asu.edu/~mabbasi2/

              Arizona State University
Data Mining and Machine Learning Lab
                                          Data Mining and Machine Learning- in a nutshell   Collective Intelligence   46

More Related Content

What's hot

Information Seeking Information Literacy: What is all this?
Information Seeking Information Literacy:  What is all this?Information Seeking Information Literacy:  What is all this?
Information Seeking Information Literacy: What is all this?Johan Koren
 
Fact Checking & Information Retrieval
Fact Checking & Information RetrievalFact Checking & Information Retrieval
Fact Checking & Information RetrievalMatthew Lease
 
計算社會科學初探- 當電腦科學家遇上社會科學
計算社會科學初探-當電腦科學家遇上社會科學計算社會科學初探-當電腦科學家遇上社會科學
計算社會科學初探- 當電腦科學家遇上社會科學Sheng-Wei (Kuan-Ta) Chen
 
Introduction What Is Information
Introduction   What Is InformationIntroduction   What Is Information
Introduction What Is Informationguestab667e
 
The Architecture of Understanding
The Architecture of UnderstandingThe Architecture of Understanding
The Architecture of UnderstandingPeter Morville
 

What's hot (6)

Information Seeking Information Literacy: What is all this?
Information Seeking Information Literacy:  What is all this?Information Seeking Information Literacy:  What is all this?
Information Seeking Information Literacy: What is all this?
 
Open Data Journalism
Open Data JournalismOpen Data Journalism
Open Data Journalism
 
Fact Checking & Information Retrieval
Fact Checking & Information RetrievalFact Checking & Information Retrieval
Fact Checking & Information Retrieval
 
計算社會科學初探- 當電腦科學家遇上社會科學
計算社會科學初探-當電腦科學家遇上社會科學計算社會科學初探-當電腦科學家遇上社會科學
計算社會科學初探- 當電腦科學家遇上社會科學
 
Introduction What Is Information
Introduction   What Is InformationIntroduction   What Is Information
Introduction What Is Information
 
The Architecture of Understanding
The Architecture of UnderstandingThe Architecture of Understanding
The Architecture of Understanding
 

Viewers also liked

La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...
La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...
La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...Pier Luca Lanzi
 
IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...
IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...
IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...Pier Luca Lanzi
 
Algoritmi e Calcolo Parallelo 2012/2013 - Algoritmi
Algoritmi e Calcolo Parallelo 2012/2013 - AlgoritmiAlgoritmi e Calcolo Parallelo 2012/2013 - Algoritmi
Algoritmi e Calcolo Parallelo 2012/2013 - AlgoritmiPier Luca Lanzi
 
Tom Malone - Program for the Future Dec. 8
Tom Malone - Program for the Future Dec. 8Tom Malone - Program for the Future Dec. 8
Tom Malone - Program for the Future Dec. 8TechVirtual
 
Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014
Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014
Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014Pier Luca Lanzi
 
DMTM 2015 - 01 Course Introduction
DMTM 2015 - 01 Course IntroductionDMTM 2015 - 01 Course Introduction
DMTM 2015 - 01 Course IntroductionPier Luca Lanzi
 

Viewers also liked (6)

La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...
La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...
La "Scienza" dei Video Giochi - Seminari Cultura Matematica - Politecnico di ...
 
IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...
IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...
IEEE Game Innovation Conference 2013 - Tuning Mobile Game Design Using Data M...
 
Algoritmi e Calcolo Parallelo 2012/2013 - Algoritmi
Algoritmi e Calcolo Parallelo 2012/2013 - AlgoritmiAlgoritmi e Calcolo Parallelo 2012/2013 - Algoritmi
Algoritmi e Calcolo Parallelo 2012/2013 - Algoritmi
 
Tom Malone - Program for the Future Dec. 8
Tom Malone - Program for the Future Dec. 8Tom Malone - Program for the Future Dec. 8
Tom Malone - Program for the Future Dec. 8
 
Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014
Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014
Introduction to Procedural Content Generation - Codemotion 29 Novembre 2014
 
DMTM 2015 - 01 Course Introduction
DMTM 2015 - 01 Course IntroductionDMTM 2015 - 01 Course Introduction
DMTM 2015 - 01 Course Introduction
 

Similar to Collective Inteligence Part I

Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?
Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?
Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?Katina Michael
 
Real-World Behavior Analysis through a Social Media Lens
Real-World Behavior Analysis through a Social Media LensReal-World Behavior Analysis through a Social Media Lens
Real-World Behavior Analysis through a Social Media LensAli Abbasi
 
Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...
Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...
Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...Paul Gilbreath
 
The Web and the Collective Intelligence - How to use Collective Intelligence ...
The Web and the Collective Intelligence - How to use Collective Intelligence ...The Web and the Collective Intelligence - How to use Collective Intelligence ...
The Web and the Collective Intelligence - How to use Collective Intelligence ...Hélio Teixeira
 
Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...
Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...
Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...Artificial Intelligence Institute at UofSC
 
Big and Small Web Data
Big and Small Web DataBig and Small Web Data
Big and Small Web DataMarieke Guy
 
Learning To Recognize Reliable Users And Content In Social Media With Coupled...
Learning To Recognize Reliable Users And Content In Social Media With Coupled...Learning To Recognize Reliable Users And Content In Social Media With Coupled...
Learning To Recognize Reliable Users And Content In Social Media With Coupled...Ali Abbasi
 
datamining_Lecture_1(introduction).pptx
datamining_Lecture_1(introduction).pptxdatamining_Lecture_1(introduction).pptx
datamining_Lecture_1(introduction).pptxHASHEMHASH
 
SOCIAM: The Theory and Practice of Social Machines
SOCIAM: The Theory and Practice of Social MachinesSOCIAM: The Theory and Practice of Social Machines
SOCIAM: The Theory and Practice of Social MachinesSOCIAM Project
 
SPAWAR/N-NC PLTW/STEM brief 20130620
SPAWAR/N-NC PLTW/STEM brief 20130620SPAWAR/N-NC PLTW/STEM brief 20130620
SPAWAR/N-NC PLTW/STEM brief 20130620N/NC - SPAWAR
 
Machine Learning and Artificial Intelligence
Machine Learning and Artificial IntelligenceMachine Learning and Artificial Intelligence
Machine Learning and Artificial IntelligenceV.N.S. Satya Teja
 
Practical Applications of Visual Analytics
Practical Applications of Visual AnalyticsPractical Applications of Visual Analytics
Practical Applications of Visual AnalyticsTeradata Aster
 
Introduction to Artificial Intelligence and Machine Learning
Introduction to Artificial Intelligence and Machine Learning Introduction to Artificial Intelligence and Machine Learning
Introduction to Artificial Intelligence and Machine Learning Emad Nabil
 
Data sharing in the age of the Social Machine
Data sharing in the age of the Social MachineData sharing in the age of the Social Machine
Data sharing in the age of the Social MachineUlrik Lyngs
 
Human and Machine Learning
Human and Machine LearningHuman and Machine Learning
Human and Machine LearningAnkur Jain
 
Artificial intelligence Ch1
Artificial intelligence Ch1Artificial intelligence Ch1
Artificial intelligence Ch1kassatilahun
 

Similar to Collective Inteligence Part I (20)

Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?
Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?
Artificial Intelligence & Machine Learning. Is it Planet Saving Tech?
 
Real-World Behavior Analysis through a Social Media Lens
Real-World Behavior Analysis through a Social Media LensReal-World Behavior Analysis through a Social Media Lens
Real-World Behavior Analysis through a Social Media Lens
 
Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...
Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...
Web 2.0 Collective Intelligence - How to use collective intelligence techniqu...
 
The Web and the Collective Intelligence - How to use Collective Intelligence ...
The Web and the Collective Intelligence - How to use Collective Intelligence ...The Web and the Collective Intelligence - How to use Collective Intelligence ...
The Web and the Collective Intelligence - How to use Collective Intelligence ...
 
Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...
Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...
Semantic, Cognitive and Perceptual Computing -Keynote artificial intelligence...
 
Big and Small Web Data
Big and Small Web DataBig and Small Web Data
Big and Small Web Data
 
Learning To Recognize Reliable Users And Content In Social Media With Coupled...
Learning To Recognize Reliable Users And Content In Social Media With Coupled...Learning To Recognize Reliable Users And Content In Social Media With Coupled...
Learning To Recognize Reliable Users And Content In Social Media With Coupled...
 
datamining_Lecture_1(introduction).pptx
datamining_Lecture_1(introduction).pptxdatamining_Lecture_1(introduction).pptx
datamining_Lecture_1(introduction).pptx
 
SOCIAM: The Theory and Practice of Social Machines
SOCIAM: The Theory and Practice of Social MachinesSOCIAM: The Theory and Practice of Social Machines
SOCIAM: The Theory and Practice of Social Machines
 
Computing for Human Experience [v3, Aug-Oct 2010]
Computing for Human Experience [v3, Aug-Oct 2010]Computing for Human Experience [v3, Aug-Oct 2010]
Computing for Human Experience [v3, Aug-Oct 2010]
 
SPAWAR/N-NC PLTW/STEM brief 20130620
SPAWAR/N-NC PLTW/STEM brief 20130620SPAWAR/N-NC PLTW/STEM brief 20130620
SPAWAR/N-NC PLTW/STEM brief 20130620
 
Final Report.pdf
Final Report.pdfFinal Report.pdf
Final Report.pdf
 
Machine Learning and Artificial Intelligence
Machine Learning and Artificial IntelligenceMachine Learning and Artificial Intelligence
Machine Learning and Artificial Intelligence
 
Practical Applications of Visual Analytics
Practical Applications of Visual AnalyticsPractical Applications of Visual Analytics
Practical Applications of Visual Analytics
 
Introduction to Artificial Intelligence and Machine Learning
Introduction to Artificial Intelligence and Machine Learning Introduction to Artificial Intelligence and Machine Learning
Introduction to Artificial Intelligence and Machine Learning
 
Data sharing in the age of the Social Machine
Data sharing in the age of the Social MachineData sharing in the age of the Social Machine
Data sharing in the age of the Social Machine
 
Human and Machine Learning
Human and Machine LearningHuman and Machine Learning
Human and Machine Learning
 
AI
AIAI
AI
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
Artificial intelligence Ch1
Artificial intelligence Ch1Artificial intelligence Ch1
Artificial intelligence Ch1
 

More from Ali Abbasi

Social Media Mining: An Introduction
Social Media Mining: An IntroductionSocial Media Mining: An Introduction
Social Media Mining: An IntroductionAli Abbasi
 
Data Mining: an Introduction
Data Mining: an IntroductionData Mining: an Introduction
Data Mining: an IntroductionAli Abbasi
 
Active learning
Active learningActive learning
Active learningAli Abbasi
 
Disaster Relief Using Social Media Data
Disaster Relief Using Social Media DataDisaster Relief Using Social Media Data
Disaster Relief Using Social Media DataAli Abbasi
 
Evolutionary Game Theory
Evolutionary Game TheoryEvolutionary Game Theory
Evolutionary Game TheoryAli Abbasi
 
Game Theory: an Introduction
Game Theory: an IntroductionGame Theory: an Introduction
Game Theory: an IntroductionAli Abbasi
 

More from Ali Abbasi (6)

Social Media Mining: An Introduction
Social Media Mining: An IntroductionSocial Media Mining: An Introduction
Social Media Mining: An Introduction
 
Data Mining: an Introduction
Data Mining: an IntroductionData Mining: an Introduction
Data Mining: an Introduction
 
Active learning
Active learningActive learning
Active learning
 
Disaster Relief Using Social Media Data
Disaster Relief Using Social Media DataDisaster Relief Using Social Media Data
Disaster Relief Using Social Media Data
 
Evolutionary Game Theory
Evolutionary Game TheoryEvolutionary Game Theory
Evolutionary Game Theory
 
Game Theory: an Introduction
Game Theory: an IntroductionGame Theory: an Introduction
Game Theory: an Introduction
 

Recently uploaded

Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxKatherine Villaluna
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxDr. Asif Anas
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxraviapr7
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and stepobaje godwin sunday
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptxraviapr7
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxheathfieldcps1
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxraviapr7
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational PhilosophyShuvankar Madhu
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17Celine George
 

Recently uploaded (20)

Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
Ultra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptxUltra structure and life cycle of Plasmodium.pptx
Ultra structure and life cycle of Plasmodium.pptx
 
Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptx
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
General views of Histopathology and step
General views of Histopathology and stepGeneral views of Histopathology and step
General views of Histopathology and step
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptxClinical Pharmacy  Introduction to Clinical Pharmacy, Concept of clinical pptx
Clinical Pharmacy Introduction to Clinical Pharmacy, Concept of clinical pptx
 
The basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptxThe basics of sentences session 10pptx.pptx
The basics of sentences session 10pptx.pptx
 
Prescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptxPrescribed medication order and communication skills.pptx
Prescribed medication order and communication skills.pptx
 
Philosophy of Education and Educational Philosophy
Philosophy of Education  and Educational PhilosophyPhilosophy of Education  and Educational Philosophy
Philosophy of Education and Educational Philosophy
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17How to Add Existing Field in One2Many Tree View in Odoo 17
How to Add Existing Field in One2Many Tree View in Odoo 17
 

Collective Inteligence Part I

  • 1. DATA MINING AND MACHINE LEARNING IN A NUTSHELL COLLECTIVE INTELLIGENCE PART I Mohammad-Ali Abbasi http://www.public.asu.edu/~mabbasi2/ SCHOOL OF COMPUTING, INFORMATICS, AND DECISION SYSTEMS ENGINEERING ARIZONA STATE UNIVERSITY Arizona State University http://dmml.asu.edu/ Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 1
  • 2. About Collective Intelligence • Definition of collective intelligence – Examples happening around us • What constitutes collective intelligence – Groups, number of members, variety, etc. • How can one improve collective intelligence – What are necessary conditions to achieve CI – A case in data mining and machine learning? • What can one do with collective intelligence in the age of social media – Opportunities for Data Mining Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 2
  • 3. Definitions for Collective intelligence • Wikipedia – Collective intelligence is a shared or group intelligence that emerges from the collaboration and competition of many individuals • MIT Center for CI – Groups of individuals doing things collectively that seem intelligent • Toby Segaran in Programming CI – Combining the behavior, preferences, or ideas of a group of people to create novel insights • Unknown – Collective intelligence is any intelligence that arises from - or is a capacity or characteristic of - groups and other collective living systems Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 3
  • 4. Examples of collective intelligence - Wikipedia • Wikipedia • Thousands of contributors from across the world have collectively created the world’s largest encyclopedia • with almost no centralized control Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 4
  • 5. Examples of collective intelligence - PageRank • PageRank Algorithm used by Google Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 5
  • 6. Examples of collective intelligence - CAPTCHA • CAPTCHA – Completely Automated Public Turing test to tell Computers and Humans Apart – A reverse Turing test (machine to human instead of human to machine) • A service that helps to digitize books, newspapers and old time radio shows – About 200 million CAPTCHAs are solved by humans around the world every day – More than 150,000 hours of work each day Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 6
  • 7. Vark.com 1. Send a question 2. Aardvark finds the perfect person to answer 3. Get their response Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 7
  • 8. Kasparov vs. the World • Kasparov v. the World was a chess match held in 1999, when world champion Gary Kasparov played against “the World,” with the World’s moves determined by majority vote over the Internet of anyone who wanted to participate. Kasparov eventually won, but he said it was the hardest game he ever played Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 8
  • 9. Examples of collective intelligence - Threadless • Threadless.com • In Threadless, anyone who wants to can design a T- shirt, submit that design to a weekly contest, and vote for their favorite designs • the company harnesses the collective intelligence of a community of over 500,000 people to design and select T-shirts Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 9
  • 10. Examples of collective intelligence –Google Image Labeler • It is a feature, in the form of a game, of Google Image Search that allows the user to label random images to help improve the quality of Google's image search results Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 10
  • 11. Examples of collective intelligence – Ant Societies • Ant societies exhibit more intelligence than any other animal except for humans, if we measure intelligence in terms of technology. Ant societies are able to do agriculture, in fact, in several different forms of agriculture. Some ant societies keep livestock of various forms, for example, some ants keep and care for aphids for "milking”; Leaf cutters care for fungi and carry leaves to feed the fungi. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 11
  • 12. Examples of collective intelligence - Games • Games such as WorldCraft, The Sims, Halo or Second Life are designed to be more non- linear and depend on collective intelligence for expansion. • This way of sharing is gradually evolving and influencing the mindset of the current and future generations. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 12
  • 13. Principals of Collective Intelligence • Collective intelligence is of mass collaboration. In order for collective intelligence to emerge, four principles exist to promote creativity: – Openness – Peering – Sharing and – Acting globally Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 15
  • 14. Openness • Traditionally, people and companies are naturally reluctant to share ideas and intellectual property because these resources provide the edge over competitors. • However, in time, openness is promoted when people and companies began to loosen hold over these resources as they reap more benefits in doing so. • Openness enables products to gain significant improvement and scrutiny through transparent collaboration. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 16
  • 15. Peering • A form of horizontal organization with the capacity to create information technology and physical products. • One example is the ‘opening up’ of the Linux program where users are free to modify and develop it provided that they made it available for others. • Participants in this form of collective intelligence may have different motivations for contributing, but the results achieved are for the improvement of a product or service. • “Peering succeeds because it leverages self-organization – a style of production that works more effectively than hierarchical management for certain tasks.” Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 17
  • 16. Sharing • Research has shown that more and more companies have started to share some, while maintaining some degree of control over others, like potential and critical patent rights. • This is because companies have realized that by limiting all their intellectual property, they are shutting out all possible opportunities. • Sharing some has allowed them to expand their market and bring out products faster. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 18
  • 17. Acting Globally • The advancement in communication technology has prompted the rise of global companies, or e- Commerce that has allowed individuals to set up businesses at low to almost no overhead costs. • The influence of the Internet is widespread, therefore a globally integrated company would have no geographical boundaries but have global connections, allowing them to gain access to new markets, ideas and technology. • Therefore it is important for firms to get updated and remain globally competitive or they will face a declining rate of clients. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 19
  • 18. Types of Collective Intelligence Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 20
  • 19. Elements of Collective Intelligence • Staffing – Who is performing the task? • Incentives – Why are they doing it? • Goal – What is being accomplished? • Structure, process – How is it being done? Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 21
  • 20. Elements of Collective Intelligence • Who? – Hierarchy – Crowd • Why? – Money – Love – Glory • What? – Create – Decide • Who – Collection – Collaboration Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 22
  • 21. Mapping the collective intelligence elements for Wikipedia Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 23
  • 22. Issues with Crowd Wisdom • Questions – Why can the crowd be smarter than any individual in the crowd? – Is it guaranteed? If not, what are the conditions under which the crowd can make best decisions? – How can one gauge the reliability of crowd wisdom? Is crowd wisdom valid, trustworthy, and verifiable? – How to find a crowd, its leader/influencer/average opinion? – How is each member influenced by others? Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 24
  • 23. Collective Intelligence and Societies • The main base of all kinds of CI’s is society • CI in traditional societies – Families, companies, countries, and armies are all groups of individuals doing things collectively that, at least sometimes, seem intelligent • CI in Web based societies- Social Networking sites – Internet and specially Web 2.0 applications provide a platform for communications and building societies Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 25
  • 24. Collective Intelligence and the Internet • Web 2.0 • Social Computing Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 26
  • 25. Web Impacts on CI • The ability of new media to easily store and retrieve information, predominantly through databases and the Internet, allows it to be shared without difficulty. • Thus, through interaction with new media, knowledge easily passes between sources resulting in another form of collective intelligence Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 27
  • 26. WEB 2.0 and Many Variants Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 28
  • 27. Elements of WEB 2.0 Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 29
  • 28. Web 2.0: Evolution Towards a Read/Write Platform Web 1.0 Web 2.0 (1993-2003) (2003- beyond) Pretty much HTML pages viewed through a Web pages, plus a lot of other “content” shared browser over the web, with more interactivity; more like an application than a “page” “Read” Mode “Write” & Contribute “Page” Primary Unit of “Post / record” content “static” State “dynamic” Web browser Viewed through… Browsers, RSS Readers, anything “Client Server” Architecture “Web Services” Web Coders Content Created by… Everyone “geeks” Domain of… “mass amatuerization” Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 30
  • 29. CI in Social Media • Crowd members assign different weights to individual inputs on the basis of their relationship with the people who provided them and then make individual decisions – Blogosphere – Facebook – YouTube – Epinions.com – Amazon – eBay – Digg Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 31
  • 30. Blogging is the Most Recognized Example of Web 2.0 Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 32
  • 31. Blogging is the Most Recognized Example of Web 2.0 Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 33
  • 32. Wikipedia is a Collaborative Dictionary Being Edited in Real-time by Anyone Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 34
  • 33. Alive At ASU Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 35
  • 34. WEB 2.0 Technologies • APIs • RSS (Really Simple Syndication) – Content Syndication • Web Services – Open Data • AJAX (Asynchronous Javascript and XML) • CSS (Cascading Style Sheets) – Content with Style Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 36
  • 35. WEB 2.0, Summing Up • Web 2.0 hard to define, but very far from just hype – Culmination of a number of Web trends • Importance of Open Data – Allows communities to assemble unique tailored applications • Importance of Users – Seek and create network effects • Browser as Application Platform – Huge potential for new kinds of Web applications Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 37
  • 36. Programming Collective Intelligence Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 38
  • 37. Crawl the web Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 39
  • 38. Spiders (Robots/Bots/Crawlers) • Start with a comprehensive set of root URL’s from which to start the search. • Follow all links on these pages recursively to find additional pages. • Index all novel found pages in an inverted index as they are encountered. • May allow users to directly submit pages to be indexed (and crawled from). Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 40
  • 39. Search Strategies Breadth-first Search Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 41
  • 40. Search Strategies (cont) Depth-first Search Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 42
  • 41. Search Strategy Trade-Off’s • Breadth-first explores uniformly outward from the root page but requires memory of all nodes on the previous level (exponential in depth). Standard spidering method. • Depth-first requires memory of only depth times branching-factor (linear in depth) but gets “lost” pursuing a single thread. • Both strategies implementable using a queue of links (URL’s). Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 43
  • 42. Spidering Algorithm • Initialize queue (Q) with initial set of known URL’s. • Until Q empty or page or time limit exhausted: – Pop URL, L, from front of Q. – If L is not to an HTML page (.gif, .jpeg, .ps, .pdf, .ppt…) • continue loop. – If already visited L, continue loop. – Download page, P, for L. – If cannot download P (e.g. 404 error, robot excluded) • continue loop. – Index P (e.g. add to inverted index or store cached copy). – Parse P to obtain list of new links N. – Append N to the end of Q. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 44
  • 43. Keeping Spidered Pages Up to Date • Web is very dynamic: many new pages, updated pages, deleted pages, etc. • Periodically check spidered pages for updates and deletions: – Just look at header info (e.g. META tags on last update) to determine if page has changed, only reload entire page if needed. • Track how often each page is updated and preferentially return to pages which are historically more dynamic. • Preferentially update pages that are accessed more often to optimize freshness of more popular pages. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 45
  • 44. Mohammad-Ali Abbasi (Ali), Ali, is a Ph.D student at Data Mining and Machine Learning Lab, Arizona State University. His research interests include Data Mining, Machine Learning, Social Computing, and Social Media Behavior Analysis. http://www.public.asu.edu/~mabbasi2/ Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 46

Editor's Notes

  1. In Wikipedia, thousands of contributors from across the world have collectively created the world’s largest encyclopedia, with articles of remarkably high quality. Wikipedia has been developed with almost no centralized control. Anyone who wants to can change almost anything, and decisions about what changes to keep are made by a loose consensus of those who care. What’s more, the people who do all this work don’t even get paid; they’re volunteers.
  2. Google, for instance, takes the judgments made by millions of people as they create links to Web pages and harnesses that collective knowledge of the entire Web to produce amazingly intelligent answers to the questions we type into the Google search bar.
  3. http://en.wikipedia.org/wiki/CAPTCHACAPTCHA is vulnerable to a relay attack that uses humans to solve the puzzles. One approach involves relaying the puzzles to a group of human operators who can solve CAPTCHAs. In this scheme, a computer fills out a form and when it reaches a CAPTCHA, it gives the CAPTCHA to the human operator to solve.
  4. Yahoo!Answers is another example.
  5. In Threadless, anyone who wants to can design a T-shirt, submit that design to a weekly contest, and vote for their favorite designs. From the entries receiving the most votes, the company selects winning designs, puts them into production, and gives prizes and royalties to the winning designers. In this way, the company harnesses the collective intelligence of a community of over 500,000 people to design and select T-shirts
  6. Turtles, Termites, and Traffic Jams - Explorations in Massively Parallel Microworlds
  7. More examples on collective intelligence can be seen in games.
  8. http://en.wikipedia.org/wiki/Collective_intelligence
  9. Allowing others to share ideas and bid for franchising
  10. This principle has been controversial with the question being “Should there be a law against the distribution of intellectual property?”
  11. Online talent scouts pay off – USA Today April 1, 2010 Money Section B by John Swartzhttp://www.usatoday.com/money/industries/technology/2010-04-01-crowdsourcing01_ST_N.htm
  12. http://en.wikipedia.org/wiki/File:CI_types1s.jpg
  13. http://en.wikipedia.org/wiki/Crowd_wisdom