David De Roure
@dder
Intersection, Scale, and
Social Machines:
Scholarship in the digital world
DIRECTOR, UNIVERSITY OF OXFORD E-RESEARCH CENTRE

Today we are witnessing several shifts in scholarly practice,
in and across multiple disciplines, as researchers embrace
digital techniques to tackle established research questions in
new ways and new questions afforded by digital and
digitized collections, approaches, and technologies.
Pervasive adoption of technology, coupled with the co-
creation of new social processes, has created a new and
complex space for scholarship where citizens both generate
and analyze data as they interact at the intersection of the
physical and digital.
Drawing on a background in distributed computing, and
adopting the lens of Social Machines, this talk discusses
current activity in digital scholarship, framing it in its
interdisciplinary settings.
data-intensive
social processes
social machines

The Big Picture(s)
Challenging Assumptions

ChristineBorgman

New Forms of Data
▶ Internet data, derived from social
media and other online interactions
(including data gathered by
connected people and devices, eg
mobile devices, wearable
technology, Internet of Things)
▶ Tracking data, monitoring the
movement of people and objects
(including GPS/geolocation data,
traffic and other transport sensor
data, CCTV images etc)
▶ Satellite and aerial imagery (eg
Google Earth, Landsat, infrared,
radar mapping etc) http://www.oecd.org/sti/sci-tech/new-data-for-
understanding-the-human-condition.htm

The
Big
Picture
More people
Moremachines
Big Data
Big Compute
Conventional
Computation
“Big Social”
Social Networks
e-infrastructure
Online R&D
(Science 2.0)
Social
Machines
@dder

theODI.org

Data Detect Store AnalyticsFilter Analysts
@dder

There is no such thing as the Internet of Things
There is no such thing as a closed system
Humans are creative and subversive
The Rise of the Bots A Swarm of Drones
Accidents happen (in the lab, bin)
Holding machines to account Software vulnerability
Where are the throttle points?
@dder

F i r s t

New Research Questions
▶ Social media data offers
the possibility of studying
social processes as they
unfold at the level of
populations, as an
alternative to traditional
surveys or interviews.
▶ The data from social media
is described as "qualitative
data on a quantitative
scale" and requires
innovative analysis
techniques.
Social media
data and real
time
analytics

Edwards, P. N., et al. (2013) Knowledge Infrastructures: Intellectual Frameworks and
Research Challenges. Ann Arbor: Deep Blue. http://hdl.handle.net/2027.42/97552

Social Machines
Empowered Citizens

Social
Machines
Defini6on
TBL
Pip Willcox

https://twitter.com/CR_UK/status/446223117841494016/
Some people's smartphones
had autocorrected the word
"BEAT" to instead read
"BEAR".
"Thank you for choosing an
adorable polar bear," the
reply from the WWF said.
"We will call you today to set
up your adoption."
http://www.bbc.com/news/technology-26723457

SOCIAM: The Theory and Practice of Social Machines is funded by the UK Engineering and Physical Sciences Research Council
(EPSRC) under grant number EPJ017728/1 and comprises the Universities of Southampton, Oxford and Edinburgh. See sociam.org

“Yet
Wikipedia
and
its
stated
ambi6on
to
“compile
the
sum
of
all
human
knowledge”
are
in
trouble.
The
volunteer
workforce
that
built
the
project’s
flagship,
the
English-­‐language
Wikipedia—and
must
defend
it
against
vandalism,
hoaxes,
and
manipula6on—
has
shrunk
by
more
than
a
third
since
2007
and
is
s6ll
shrinking…
The
main
source
of
those
problems
is
not
mysterious.
The
loose
collec6ve
running
the
site
today,
es6mated
to
be
90
percent
male,
operates
a
crushing
bureaucracy
with
an
oSen
abrasive
atmosphere
that
deters
newcomers
who
might
increase
par6cipa6on
in
Wikipedia
and
broaden
its
coverage…”
http://www.technologyreview.com/featuredstory/520446/the-decline-of-wikipedia/

Scientists
Talk
Forum
Image
Classification
data reduction
Citizen Scientists

“Panoptes has been designed so
that it’s easier for us to update
and maintain, and to allow
more powerful tools for project
builders. It’s also open source
from the start, and if you find
bugs or have suggestions about
the new site you can note them
on Github (or, if you’re so
inclined, contribute to the
codebase yourself). “
"
http://blog.zooniverse.org/2015/06/29/a-whole-new-zooniverse/
http://monsterspedia.wikia.com/wiki/File:Argus-Panoptes.jpg
Panoptes

Musical Social Machines
Social Machines of Scholarship

INT. VERSE VERSE VERSE VERSEBRIDGEBRIDGE OUT.
ê
The
Problem
signal
understanding

salami.music.mcgill.ca
Jordan B. L. Smith, J. Ashley Burgoyne, Ichiro Fujinaga, David De Roure, and J.
Stephen Downie. 2011. Design and creation of a large-scale database of structural
annotations. In Proceedings of the International Society for Music Information
Retrieval Conference, Miami, FL, 555–60

Digital
Music
Collec6ons
Student-­‐sourced
ground
truth
Community
SoSware
Linked
Data
Repositories
Supercomputer
23,000 hours of
recorded music
Music Information
Retrieval Community
SALAMI

Ashley Burgoyne

class structure
Ontology models properties from musicological domain
• Independent of Music Information Retrieval research and
signal processing foundations
• Maintains an accurate and complete description of
relationships that link them
Segment
Ontology
Ben Fields, Kevin Page, David De Roure and Tim Crawford (2011) "The Segment
Ontology: Bridging Music-Generic and Domain-Specific" in 3rd International
Workshop on Advances in Music Information Research (AdMIRe 2011) held in
conjunction with IEEE International Conference on Multimedia and Expo (ICME),
Barcelona, July 2011

www.music-ir.org/mirex
Music Information Retrieval Evaluation eXchange
Audio Onset Detection
Audio Beat Tracking
Audio Key Detection
Audio Downbeat Detection
Real-time Audio to Score Alignment(a.k.a
Score Following)
Audio Cover Song Identification
Discovery of Repeated Themes & Sections
Audio Melody Extraction
Query by Singing/Humming
Audio Chord Estimation
Singing Voice Separation
Audio Fingerprinting
Music/Speech Classification/Detection
Audio Offset Detection
Downie, J. Stephen, Andreas F. Ehmann, Mert Bay and M. Cameron Jones. (2010).
The Music Information Retrieval Evaluation eXchange: Some Observations and
Insights. Advances in Music Information Retrieval Vol. 274, pp. 93-115

seasr.org/meandre
Meandre

Stephen
Downie

http://chordify.net/

Studying Social Machines
Scholarship of Social Machines

Ecosystem
Perspective
• We see a community of
living, hybrid organisms,
rather than a set of
machines which happen to
have humans amongst
their components
• Their successes and
failures inform the design
and construction of their
offspring and successors

time
Social Machine instances
@dder

Observer of
one social
machine
Observers using third
party observatory
Observer of
multiple social
machines
Human
participants in
Social
Machine
Human participants in
multiple Social Machines
Observer of Social
Machine infrastructure
1
4
2
3
5
6
SM
SM
SM
Social Machine
Observing Social
Machines
7
@dder
De Roure, D.,
Hooper, C., Page,
K., Tarte, S., and
Willcox, P. 2015.
Observing Social
Machines Part 2:
How to Observe?
ACM Web Science

The Web
Observatory
Tiropanis, T., Hall, W., Shadbolt, N., De Roure, D.,
Contractor, N. and Hendler, J. 2013. The Web Science
Observatory, IEEE Intelligent Systems 28(2) pp 100–104.

Simpson, R., Page, K.R. and
De Roure, D. 2014.
Zooniverse: observing the
world's largest citizen science
platform. In Proceedings of
the companion publication of
the 23rd international
conference on World Wide
Web, 1049-1054.

By Ségolène Tarte, David De Roure
and Pip Willcox
Working out the Plot
The Role of Stories in
Social Machines
Tarte, S.M., De Roure, D. and Willcox, P. 2014. Working out the Plot: the Role of
Stories in Social Machines. SOCM2014: The Theory and Practice of Social
Machines, Seoul, Korea, International World Wide Web Conferences pp. 909–914

STORYTELLING AS A STETHOSCOPE
FOR SOCIAL MACHINES
1. Sociality through storytelling potential
and realization
2. Sustainability through reactivity and
interactivity
3. Emergence through collaborative
authorship and mixed authority
Zooniverse
is
a
highly
storified
Social
Machine
Facebook
doesn’t
allow
for
improvisa6on
Wikipedia
assigns
authority
rights
rigidly
http://ora.ox.ac.uk/objects/ora:8033

Pip Willcox

Tarte, S. Willcox, P., Glaser, H. and De Roure, D. 2015. Archetypal Narratives in Social
Machines: Approaching Sociality through Prosopography. ACM Web Science 2015.
SégolèneTarte

Automation
Dystopias

https://www.gartner.com/technology/research/digital-marketing/transit-map.jsp

Notifications and automatic re-runs
Machines are users too
Autonomic
Curation
Self-repair
New research?

The challenge is to foster the co-constituted socio-technical system
on the right i.e. a computationally-enabled sense-making network of
expertise, data, software, models, and narratives.
Big data elephant versus sense-making network?

The
R
Dimensions
Research
Objects
facilitate
research
that
is
reproducible,
repeatable,
replicable,
reusable,
referenceable,
retrievable,
reviewable,
replayable,
re-­‐interpretable,
reprocessable,
recomposable,
reconstructable,
repurposable,
reliable,
respec`ul,
reputable,
revealable,
recoverable,
restorable,
reparable,
refreshable?”
@dder 14 April 2014
sci
method
access
understand
new
use
social
cura6on
Research
Object
Principles
De Roure, D. 2014. The future
of scholarly communications.
Insights: the UKSG journal,
27, (3), 233-238.
DOI 10.1629/2048-7754.171

Closing thoughts
Reflections

Principles of Robotics
1. Robots are multi-use tools. Robots should not be designed solely
or primarily to kill or harm humans, except in the interests of
national security.
2. Humans, not robots, are responsible agents. Robots should be
designed; operated as far as is practicable to comply with existing
laws & fundamental rights & freedoms, including privacy.
3. Robots are products. They should be designed using processes
which assure their safety and security.
4. Robots are manufactured artefacts. They should not be designed
in a deceptive way to exploit vulnerable users; instead their
machine nature should be transparent.
5. The person with legal responsibility for a robot should be
attributed.
https://www.epsrc.ac.uk/research/ourportfolio/themes/engineering/activities/principlesofrobotics/
AlanWinfield

Principles of Robotics Social Machines?
1. Social Machines are multi-use tools. Social Machines should not
be designed solely or primarily to kill or harm humans, except in
the interests of national security.
2. Humans, not Social Machines, are responsible agents. Social
Machines should be designed; operated as far as is practicable to
comply with existing laws & fundamental rights & freedoms,
including privacy.
3. Social Machines are products. They should be designed using
processes which assure their safety and security.
4. Social Machines are manufactured artefacts. They should not be
designed in a deceptive way to exploit vulnerable users; instead
their machine nature should be transparent.
5. The person with legal responsibility for a Social Machine should
be attributed.

http://groups.csail.mit.edu/mac/projects/amorphous/6.966/

Normal Science – computer science is a
puzzle-solving activity under our current
paradigm, inspired by great achievements.
Kuhn cycle
We are in the period of crisis, where the failure of established
methods permits us to experiment with new methods to crack the
anomaly. We experiment with social machines as an underpinning
model.
If successful, social machines become the new paradigm and
scientific revolution has occurred. This is evidenced by the papers and
books that train the next generation.
De Roure, D. 2014. The
Emerging Paradigm of Social
Machines, Digital
Enlightenment Yearbook 2014
227 K. O’Hara et al. (Eds.)
IOS Press, 2014. pp 227-234.
Successful social machines, like Wikipedia,
are the anomaly. They do not yield to
standard techniques despite attempts to
extend those techniques and fit social
machines in as machines. cf Newtonian
mechanics.

Data-Intensive
+
Social Processes
co-created real-time at-scale
=
Social Machines

PipWillcox

david.deroure@oerc.ox.ac.uk
@dder
Thanks to Stephen Downie, Ich Fujinaga, Chris Lintott,
Grant Miller, Kevin Page, Ségolène Tarte, Pip Willcox,
Project SOCIAM and Project MAC.
http://www.slideshare.net/davidderoure/scholarship-in-the-digital-world
Supported by SOCIAM: The Theory and Practice of Social Machines, funded by the UK Engineering and Physical
Sciences Research Council (EPSRC), under grant number EP/J017728/1, also FAST EP/L019981/1, and Smart
Society: Hybrid and Diversity-Aware Collective Adaptive Systems: When People Meet Machines to Build a Smarter
Society, funded under the European Commission FP7-ICT FET Proactive Initiative: Fundamentals of Collective
Adaptive Systems (FOCAS), Project Reference 600854.

www.oerc.ox.ac.uk
david.deroure@oerc.ox.ac.uk
@dder