How to Add Existing Field in One2Many Tree View in Odoo 17
Collective Inteligence Part I
1. DATA MINING AND MACHINE LEARNING
IN A NUTSHELL
COLLECTIVE INTELLIGENCE
PART I
Mohammad-Ali Abbasi
http://www.public.asu.edu/~mabbasi2/
SCHOOL OF COMPUTING, INFORMATICS, AND DECISION SYSTEMS ENGINEERING
ARIZONA STATE UNIVERSITY
Arizona State University
http://dmml.asu.edu/
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 1
2. About Collective Intelligence
• Definition of collective intelligence
– Examples happening around us
• What constitutes collective intelligence
– Groups, number of members, variety, etc.
• How can one improve collective intelligence
– What are necessary conditions to achieve CI
– A case in data mining and machine learning?
• What can one do with collective intelligence in
the age of social media
– Opportunities for Data Mining
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 2
3. Definitions for Collective intelligence
• Wikipedia
– Collective intelligence is a shared or group intelligence that
emerges from the collaboration and competition of many
individuals
• MIT Center for CI
– Groups of individuals doing things collectively that seem
intelligent
• Toby Segaran in Programming CI
– Combining the behavior, preferences, or ideas of a group
of people to create novel insights
• Unknown
– Collective intelligence is any intelligence that arises from -
or is a capacity or characteristic of - groups and other
collective living systems
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 3
4. Examples of collective intelligence - Wikipedia
• Wikipedia
• Thousands of contributors from across the
world have collectively created the world’s
largest encyclopedia
• with almost no centralized control
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 4
5. Examples of collective intelligence - PageRank
• PageRank Algorithm used by Google
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 5
6. Examples of collective intelligence - CAPTCHA
• CAPTCHA
– Completely Automated Public Turing test to tell Computers and Humans Apart
– A reverse Turing test (machine to human instead of human to machine)
• A service that helps to digitize books,
newspapers and old time radio shows
– About 200 million CAPTCHAs are solved by
humans around the world every day
– More than 150,000 hours of work each day
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 6
7. Vark.com
1. Send a question
2. Aardvark finds the perfect person to answer
3. Get their response
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 7
8. Kasparov vs. the World
• Kasparov v. the World was a chess match held
in 1999, when world champion Gary Kasparov
played against “the World,” with the World’s
moves determined by majority vote over the
Internet of anyone who wanted to participate.
Kasparov eventually won, but he said it
was the hardest game he ever played
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 8
9. Examples of collective intelligence - Threadless
• Threadless.com
• In Threadless, anyone who
wants to can design a T-
shirt, submit that design
to a weekly contest, and
vote for their favorite
designs
• the company harnesses
the collective intelligence
of a community of over
500,000 people to design
and select T-shirts
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 9
10. Examples of collective intelligence –Google Image
Labeler
• It is a feature, in the form of a game,
of Google Image Search that allows the user to
label random images to help improve the
quality of Google's image search results
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 10
11. Examples of collective intelligence – Ant Societies
• Ant societies exhibit more intelligence than
any other animal except for humans, if we
measure intelligence in terms of technology.
Ant societies are able to do agriculture, in fact,
in several different forms of agriculture. Some
ant societies keep livestock of various forms,
for example, some ants keep and care for
aphids for "milking”; Leaf cutters care for
fungi and carry leaves to feed the fungi.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 11
12. Examples of collective intelligence - Games
• Games such as WorldCraft, The Sims, Halo or
Second Life are designed to be more non-
linear and depend on collective intelligence
for expansion.
• This way of sharing is gradually evolving and
influencing the mindset of the current and
future generations.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 12
13. Principals of Collective Intelligence
• Collective intelligence is of mass collaboration.
In order for collective intelligence to emerge,
four principles exist to promote creativity:
– Openness
– Peering
– Sharing and
– Acting globally
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 15
14. Openness
• Traditionally, people and companies are naturally
reluctant to share ideas and intellectual property
because these resources provide the edge over
competitors.
• However, in time, openness is promoted when
people and companies began to loosen hold over
these resources as they reap more benefits in
doing so.
• Openness enables products to gain significant
improvement and scrutiny through transparent
collaboration.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 16
15. Peering
• A form of horizontal organization with the capacity to
create information technology and physical products.
• One example is the ‘opening up’ of the Linux program
where users are free to modify and develop it provided that
they made it available for others.
• Participants in this form of collective intelligence may have
different motivations for contributing, but the results
achieved are for the improvement of a product or service.
• “Peering succeeds because it leverages self-organization – a
style of production that works more effectively than
hierarchical management for certain tasks.”
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 17
16. Sharing
• Research has shown that more and more
companies have started to share some, while
maintaining some degree of control over
others, like potential and critical patent rights.
• This is because companies have realized that
by limiting all their intellectual property, they
are shutting out all possible opportunities.
• Sharing some has allowed them to expand
their market and bring out products faster.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 18
17. Acting Globally
• The advancement in communication technology
has prompted the rise of global companies, or e-
Commerce that has allowed individuals to set up
businesses at low to almost no overhead costs.
• The influence of the Internet is widespread,
therefore a globally integrated company would
have no geographical boundaries but have global
connections, allowing them to gain access to new
markets, ideas and technology.
• Therefore it is important for firms to get updated
and remain globally competitive or they will face
a declining rate of clients.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 19
18. Types of Collective Intelligence
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 20
19. Elements of Collective Intelligence
• Staffing
– Who is performing the task?
• Incentives
– Why are they doing it?
• Goal
– What is being accomplished?
• Structure, process
– How is it being done?
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 21
20. Elements of Collective Intelligence
• Who?
– Hierarchy
– Crowd
• Why?
– Money
– Love
– Glory
• What?
– Create
– Decide
• Who
– Collection
– Collaboration
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 22
21. Mapping the collective intelligence elements for
Wikipedia
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 23
22. Issues with Crowd Wisdom
• Questions
– Why can the crowd be smarter than any individual
in the crowd?
– Is it guaranteed? If not, what are the conditions
under which the crowd can make best decisions?
– How can one gauge the reliability of crowd
wisdom? Is crowd wisdom valid, trustworthy, and
verifiable?
– How to find a crowd, its leader/influencer/average
opinion?
– How is each member influenced by others?
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 24
23. Collective Intelligence and Societies
• The main base of all kinds of CI’s is society
• CI in traditional societies
– Families, companies, countries, and armies are all
groups of individuals doing things collectively that,
at least sometimes, seem intelligent
• CI in Web based societies- Social Networking
sites
– Internet and specially Web 2.0 applications
provide a platform for communications and
building societies
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 25
24. Collective Intelligence
and the Internet
• Web 2.0
• Social Computing
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 26
25. Web Impacts on CI
• The ability of new media to easily store and
retrieve information, predominantly through
databases and the Internet, allows it to be
shared without difficulty.
• Thus, through interaction with new media,
knowledge easily passes between sources
resulting in another form of collective
intelligence
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 27
26. WEB 2.0 and Many Variants
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 28
27. Elements of WEB 2.0
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 29
28. Web 2.0: Evolution Towards a Read/Write Platform
Web 1.0 Web 2.0
(1993-2003) (2003- beyond)
Pretty much HTML pages viewed through a Web pages, plus a lot of other “content” shared
browser over the web, with more interactivity; more like an
application than a “page”
“Read” Mode “Write” & Contribute
“Page” Primary Unit of “Post / record”
content
“static” State “dynamic”
Web browser Viewed through… Browsers, RSS Readers,
anything
“Client Server” Architecture “Web Services”
Web Coders Content Created by… Everyone
“geeks” Domain of… “mass amatuerization”
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 30
29. CI in Social Media
• Crowd members assign different weights to
individual inputs on the basis of their
relationship with the people who provided
them and then make individual decisions
– Blogosphere
– Facebook
– YouTube
– Epinions.com
– Amazon
– eBay
– Digg
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 31
30. Blogging is the Most Recognized Example of Web 2.0
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 32
31. Blogging is the Most Recognized Example of Web 2.0
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 33
32. Wikipedia is a Collaborative Dictionary Being Edited in Real-time by Anyone
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 34
33. Alive At ASU
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 35
34. WEB 2.0 Technologies
• APIs
• RSS (Really Simple Syndication)
– Content Syndication
• Web Services
– Open Data
• AJAX (Asynchronous Javascript and XML)
• CSS (Cascading Style Sheets)
– Content with Style
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 36
35. WEB 2.0, Summing Up
• Web 2.0 hard to define, but very far from just hype
– Culmination of a number of Web trends
• Importance of Open Data
– Allows communities to assemble unique tailored
applications
• Importance of Users
– Seek and create network effects
• Browser as Application Platform
– Huge potential for new kinds of Web applications
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 37
36. Programming
Collective Intelligence
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 38
37. Crawl the web
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 39
38. Spiders (Robots/Bots/Crawlers)
• Start with a comprehensive set of root URL’s
from which to start the search.
• Follow all links on these pages recursively to
find additional pages.
• Index all novel found pages in an inverted
index as they are encountered.
• May allow users to directly submit pages to be
indexed (and crawled from).
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 40
39. Search Strategies
Breadth-first Search
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 41
40. Search Strategies (cont)
Depth-first Search
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 42
41. Search Strategy Trade-Off’s
• Breadth-first explores uniformly outward from
the root page but requires memory of all
nodes on the previous level (exponential in
depth). Standard spidering method.
• Depth-first requires memory of only depth
times branching-factor (linear in depth) but
gets “lost” pursuing a single thread.
• Both strategies implementable using a queue
of links (URL’s).
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 43
42. Spidering Algorithm
• Initialize queue (Q) with initial set of known URL’s.
• Until Q empty or page or time limit exhausted:
– Pop URL, L, from front of Q.
– If L is not to an HTML page (.gif, .jpeg, .ps, .pdf, .ppt…)
• continue loop.
– If already visited L, continue loop.
– Download page, P, for L.
– If cannot download P (e.g. 404 error, robot excluded)
• continue loop.
– Index P (e.g. add to inverted index or store cached copy).
– Parse P to obtain list of new links N.
– Append N to the end of Q.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 44
43. Keeping Spidered Pages Up to Date
• Web is very dynamic: many new pages, updated
pages, deleted pages, etc.
• Periodically check spidered pages for updates and
deletions:
– Just look at header info (e.g. META tags on last update) to
determine if page has changed, only reload entire page if
needed.
• Track how often each page is updated and
preferentially return to pages which are historically
more dynamic.
• Preferentially update pages that are accessed more
often to optimize freshness of more popular pages.
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 45
44. Mohammad-Ali Abbasi (Ali),
Ali, is a Ph.D student at Data Mining
and Machine Learning Lab, Arizona
State University.
His research interests include Data
Mining, Machine Learning, Social
Computing, and Social Media Behavior
Analysis.
http://www.public.asu.edu/~mabbasi2/
Arizona State University
Data Mining and Machine Learning Lab
Data Mining and Machine Learning- in a nutshell Collective Intelligence 46
Editor's Notes
In Wikipedia, thousands of contributors from across the world have collectively created the world’s largest encyclopedia, with articles of remarkably high quality. Wikipedia has been developed with almost no centralized control. Anyone who wants to can change almost anything, and decisions about what changes to keep are made by a loose consensus of those who care. What’s more, the people who do all this work don’t even get paid; they’re volunteers.
Google, for instance, takes the judgments made by millions of people as they create links to Web pages and harnesses that collective knowledge of the entire Web to produce amazingly intelligent answers to the questions we type into the Google search bar.
http://en.wikipedia.org/wiki/CAPTCHACAPTCHA is vulnerable to a relay attack that uses humans to solve the puzzles. One approach involves relaying the puzzles to a group of human operators who can solve CAPTCHAs. In this scheme, a computer fills out a form and when it reaches a CAPTCHA, it gives the CAPTCHA to the human operator to solve.
Yahoo!Answers is another example.
In Threadless, anyone who wants to can design a T-shirt, submit that design to a weekly contest, and vote for their favorite designs. From the entries receiving the most votes, the company selects winning designs, puts them into production, and gives prizes and royalties to the winning designers. In this way, the company harnesses the collective intelligence of a community of over 500,000 people to design and select T-shirts
Turtles, Termites, and Traffic Jams - Explorations in Massively Parallel Microworlds
More examples on collective intelligence can be seen in games.
Allowing others to share ideas and bid for franchising
This principle has been controversial with the question being “Should there be a law against the distribution of intellectual property?”
Online talent scouts pay off – USA Today April 1, 2010 Money Section B by John Swartzhttp://www.usatoday.com/money/industries/technology/2010-04-01-crowdsourcing01_ST_N.htm