This document summarizes a research project analyzing the dissemination of problematic information on Facebook from 2016 to 2021. The researchers compiled a list of over 2,300 "fake news" domains and collected Facebook posts sharing links to those domains. They analyzed the link-sharing network between public Facebook pages/groups and domains, identifying clusters that commonly share the same domains. They also analyzed the on-sharing network between public Facebook spaces to identify communities with common interests spreading this information. The goal is to better understand the dynamics, themes, and networks involved in spreading problematic information at scale on social media over time.

1. CRICOS No.00213J
Weaponising Newssharing:
‘Fake News’ and Other Malinformation
Prof. Axel Bruns
Guest Professor, IKMZ, University of Zürich
a.bruns@qut.edu.au — a.bruns@ikmz.uzh.ch

2. Wardle, C., & Derakhshan, H. (2017).
Information Disorder: Toward an
Interdisciplinary Framework for Research and
Policy Making (DGI(2017)09). Council of
Europe. https://shorensteincenter.org/wp-
content/uploads/2017/10/Information-Disorder-
Toward-an-interdisciplinary-framework.pdf

3. CRICOS No.00213J
Case Study:
COVID/5G Conspiracy
Theories

4. CRICOS No.00213J
From the Fringes to the Mainstream:
How COVID-19 Conspiracy Theories
Spread across Social and Mainstream Media
Axel Bruns, with Edward Hurcombe and Stephen Harrington
a.bruns@qut.edu.au | @snurb_dot_info
QUT Digital Media Research Centre
Brisbane, Australia

5. CRICOS No.00213J
Infodemic
(https://www.un.org/en/un-coronavirus-communications-team/un-tackling-
%E2%80%98infodemic%E2%80%99-misinformation-and-cybercrime-covid-19)

6. CRICOS No.00213J
(https://www.facebook.com/wizkhalifa/posts/10157971300941142 )
(https://www.nytimes.com/2020/04/10/technology/coronavirus-5g-uk.html)
(https://www.communications.gov.au/departmental-news/5g-misinformation-and-covid-19)
(https://www.un.org/en/un-coronavirus-communications-team/un-tackling-
%E2%80%98infodemic%E2%80%99-misinformation-and-cybercrime-covid-19)
(https://twitter.com/debritz/status/1266574996383531008)
(https://www.centreforresponsibletechnology.org.au/pro_trump_accounts_coordinated_spre
ad_of_china_bio_weapon_covid_conspiracy_theory)

7. CRICOS No.00213J
Questions
• What claims are circulating?
• Who is spreading mis- and disinformation?
• How do the media engage with these claims?
• Are take-downs effective in reducing the dissemination of problematic content?
• Should official actors respond – and if so, how and when should they do so?

8. CRICOS No.00213J
Approach
• Facebook:
• Search for (covid,corona,virus,epidemi,pandemi) AND (5g) via CrowdTangle
• Timeframe: 1 Jan. 2020 to 12 Apr. 2020 (i.e. after the arson attacks in the UK and elsewhere)
• Limitations: only public pages, groups, verified profiles (for short, spaces); availability and use of Facebook differs
across countries; Latin alphabet; divergent local terms for the virus (e.g. koronawirus)
• 89,664 Posts, including false positives
• Media reporting:
• Search for (?i)(corona|virus|covid|epidem|pandem|wuhan|hubei) AND (?i)b(5g|fiveg|5-g|five-g)b via GDELT
• Timeframe: 1 Jan. 2020 to 12 Apr. 2020
• Limitations: only article titles and URLs; GDELT coverage; Latin alphabet; divergent local terms
• 2,812 articles, manually reviewed and coded  1,871 true positives

9. CRICOS No.00213J
Detailed Results
• Facebook:
• Bruns, A., Harrington, S., & Hurcombe, E. (2020). ‘Corona? 5G? Or Both?’: The Dynamics of COVID-19/5G
Conspiracy Theories on Facebook. Media International Australia, 177(1), 12-29.
https://journals.sagepub.com/doi/full/10.1177/1329878X20946113
• Media coverage:
• Bruns, A., Hurcombe, E., & Harrington, S. (2021). Covering Conspiracy: Approaches to Reporting the
COVID/5G Conspiracy Theory.” Digital Journalism. https://doi.org/10.1080/21670811.2021.1968921.
• Combined analysis:
• Bruns, A., Hurcombe, E., & Harrington, S. (2021). Coronavirus Conspiracy Theories: Tracing Misinformation
Trajectories from the Fringes to the Mainstream. In Communicating COVID-19: Interdisciplinary
Perspectives, eds. Monique Lewis, Kate Holland, and Eliza Govender. London: Palgrave.

10. CRICOS No.00213J
Click to edit
Master title style
Analysis

12. CRICOS No.00213J

13. CRICOS No.00213J
Phase 1: Prelude
• Generic conspiracy theories (1 to 26 Jan. 2020):
• Pre-existing claims about 5G
• Vague warnings about coming pandemics, not clearly
related to COVID-19
• Overlaps with other theories (vaccines, world
government, chemtrails, …)
• First explicit COVID/5G link claimed by French blog (20
Jan.): Wuhan as test region for Chinese 5G rollout
• Limited reach: Facebook spaces with <100,000
followers
• Media coverage: practically non-existent even in fringe
news sites

14. CRICOS No.00213J
Phase 2: The Wuhan-5G Connection Spreads
• Multilingual spread (27 Jan. to 24 Feb. 2020):
• From French anti-5G blog to German alternative medicine site
• English-language version (translation?) in K-Pop fan forum
• Further embellishments in English-language conspiracy pages and
sites: virus manufactured in Wuhan lab and activated by 5G, …
• Possible link to UK permission for Huawei to help build British 5G
network – and US criticism (27 Jan. 2020)
• Slowly growing reach: ~10% of spaces with 100k to 1m followers
• Media coverage: see phase 3

15. CRICOS No.00213J
Phase 3: Localisation and Embellishment
• Substantial diversification (25 Feb. to 11 Mar. 2020):
• Depopulation theories (feat. George Soros, Bill Gates, UN, Illuminati, Antichrist, ...)
• Alternative medicine theories (5G reduces oxygen absorption, ...)
• Virus denialism (COVID-19 does not exist, 5G is the sole cause of illness, ...)
• Adjustment of pre-existing conspiracy theories to new pandemic situation
• Multilingual spread: Romania, Czech Republic, Croatia, Italy, Spain, France, …
• Partial alignment with national lockdowns: more time for doomscrolling?
• Little change in reach: still ~70% of spaces with <10,000 followers
• Media coverage (phases 2+3): dominated by fringe US outlets
• 43 articles only, including 10 fact-check articles from technology / business news sites

16. CRICOS No.00213J
• Growing media interest (12 to 28 Mar. 2020):
• Celebrities spreading conspiracy theories (e.g. R&B singer Keri Hilson, 16 Mar.)
• Entertainment and tabloid media adopt humorous and incredulous, but uncritical tone
• Celebrities delete conspiracy posts, but celebrity reporting continues to circulate
• Spread especially in Africa and Southeast Asia
• Amplification for prominent conspiracy theorists (including politicians and journalists)
• Growing number of spaces with >10,000 followers (up to 40%)
• Media coverage: 98 articles, substantial number of entertainment and lifestyle outlets
• Majority of articles contain direct quotes of conspiracists or of celebrities’ social media posts
• But also growth in fact-checking articles
Phase 4: Celebrity Superspreaders

17. CRICOS No.00213J
• Evangelists, celebrities, arsonists (29 Mar. to 12 Apr. 2020):
• Apocalyptic French-language post in Africa and France (29 Mar., ~28.8m followers) + English versions
• Similar content in sermon by Nigerian evangelical pastor: Africa, South Korea, PNG (4 Apr., ~43.6m)
• Long African post containing collection of conspiracy theories says “Fire destroys all.” (30 Mar.)
• Spread in southern Africa, later also in UK anti-5G spaces (~7.8m)
• Whistleblower video from ‘former Vodafone UK executive’, later revealed to be evangelical
Zimbabwean pastor based in Luton (2 Apr., ~18m)
• UK boxer Amir Khan posts conspiracy video: Give Me Sport page (5 Apr., ~25.7m); Express news site
• Up to 60% of posts in spaces with >10,000 followers
• More than 60 arson attacks against 5G installations and technicians since early April (UK, later NL, …)
• Media coverage: 1,729 articles (92% of our dataset) – 35% on arson attacks, 23% on spread of
conspiracy theories, 11% on government responses, 11% on celebrity conspiracy claims
• Reduction in direct quoting of conspiracists (76% of articles are plain news reports)
Phase 5: Combustion Point
They are using the shutdown to build or put in mobile
incinerators and morgues to take all the 5G victims. When
people start dying in masses from 5G they will call it
corona virus.
All these technologies need to be destroyed to melt, so
they can't keep radiating us. We need to clip all wires and
then burn it all in massive bonfires. Fire destroys all.

18. CRICOS No.00213J

19. CRICOS No.00213J
Media Coverage

21. CRICOS No.00213J
Phases in Media Coverage
• Phase 1-3: Ignored
• Hardly any interest from leading media (somewhat more in Italy)
• Conspiracy theories a curiosity at best
• Even fringe media (InfoWars, Free Republic, Natural News) little engaged
• Reports support conspiracy theories and quote supporters
• Phase 4: Sensationalisation
• Focus on celebrities (Keri Hilson first), especially by tabloid and
entertainment media
• Humorous, sensationalist – but with direct quotes and screenshots, and
usually without real corrections or expert quotes
• Little interest from leading media and ‘real’ journalists

22. CRICOS No.00213J
Phases in Media Coverage
• Phase 5: Finally interested
• Predominant focus (57%) on arson and spread of conspiracy
theories
• Now especially in leading and local media
• More coverage (11%) of celebrities (Woody Harrelson, John
Cusack, Amir Khan, Amanda Holden, Romina Power, ...) and
evangelicals (in African media)
• Official statements (11%), takedowns on social media (8%),
statements from experts (8%) – where available
• Critical tenor, few direct quotes or support for conspiracy theories
– but also few explicit fact-checks (12%)
• But also significant national differences...

23. CRICOS No.00213J

24. CRICOS No.00213J
Conclusions

25. CRICOS No.00213J
Observations and Recommendations
• Key takeaways:
• The immediate impact of conspiracist sites is limited
• Hardcore conspiracy theorists are a problem in themselves, but have limited influence
• Celebrities can become superspreaders
• Their amplification inserts mis- and disinformation into more general conversations
• ‘Soft’ newsbeats are journalism’s weak spot
• Journalistic ethos less well developed – need to reflect on impact and act more responsibly
• Take-downs can delay dissemination – especially amongst ordinary users
• Communicative distance between mainstream and fringe groups is critically important –
need to avoid news coverage that undermines take-downs by creating more persistent copies
• The right time to respond to mis- and disinformation is… when?
• Earlier than in the case of 5G conspiracy theories, but not so early that it aids dissemination –
need to respond in ways that reach the same audiences: including in celebrity / tabloid media

26. CRICOS No.00213J
Irresponsible
Journalism

27. CRICOS No.00213J

28. CRICOS No.00213J
An Analogy?
• Dealing with viral mis- and disinformation:
• Take-downs, deplatforming, digital literacy ≈ lockdowns, quarantine, mask mandates:
it slows the spread of problematic information to vulnerable communities,
but does not solve the underlying problem itself.
• Deradicalisation ≈ vaccination:
it starves the viral event of potential carriers and superspreaders,
but it is efficient only if the vast majority of the population are protected from infection.
• Public information campaigns = public information campaigns:
in either case, clear and accurate information from public officials and other stakeholders
is crucial in ensuring community trust in these physical / informational health measures.

29. CRICOS No.00213J
This research is funded in part by the Australian Research Council projects
DP200101317 Evaluating the Challenge of ‘Fake News’ and Other
Malinformation and FT130100703 Understanding Intermedia Information
Flows in the Australian Online Public Sphere, and supported by the
Australian Research Council Centre of Excellence for Automated Decision-
Making and Society.
Facebook data are provided courtesy of CrowdTangle.
Acknowledgments

30. CRICOS No.00213J
Connectivity, Not
Disconnection

31. CRICOS No.00213J
Ready access to information
enables spread of ‘fake
news’, hyperpartisanship,
and polarisation.
(But also social connection
and community support.)
Hyperpartisans,
Hyperconnected
(https://twitter.com/bigfudge212121/status/1259317174776115201)

32. CRICOS No.00213J
The problem with an extraterrestrial-
conspiracy mailing list isn’t that it’s an echo
chamber; it’s that it thinks there’s a
conspiracy by extraterrestrials.
— David Weinberger, Salon, 21 Feb. 2004
(https://commons.wikimedia.org/wiki/File:David_Weinberger.jpg)

33. CRICOS No.00213J
This, but with a different
system of coordinates, also
drives conspiracy theories.
Weaponising
Media Literacy
(https://britannicalearn.com/blog/britannica-tackles-media-literacy/)

34. (https://theconversation.com/we-live-in-an-age-of-
fake-news-but-australian-children-are-not-learning-
enough-about-media-literacy-141371)

35. CRICOS No.00213J
‘Fake News’:
The Big Picture

36. CRICOS No.00213J
'Fake News’ on Facebook:
A Large-Scale Longitudinal Study of Problematic
Link-Sharing Practices from 2016 to 2020
Axel Bruns, Daniel Angus, Edward Hurcombe, Stephen Harrington, and Jane Tan
QUT Digital Media Research Centre
a.bruns | daniel.angus | edward.hurcombe | s.Harrington | x28.tan @ qut.edu.au
@snurb_dot_info | @antmandan | @EddyFHurc | @_StephenH | @jane_txy

37. CRICOS No.00213J
Aims
• Beyond ‘fake news’:
• Understanding the dissemination of problematic information across social media at scale
• Identifying different types of problematic information (by theme, audience, dynamics, …)
• Problematic information on Facebook:
• Dynamics of content sharing over time (key events, per domain, impact of take-downs)
• Networks of interaction and communities of interest (thematic, ideological, geographic, …)
• Funding:
• ARC Discovery project DP200101317:
Evaluating the Challenge of 'Fake News' and Other Malinformation

38. CRICOS No.00213J
Data
• FakeNIX:
• Iteratively updated masterlist of domains listed in existing studies of ‘fake news’
• 2,314 domains to date (from Shao et al., 2016; Starbird et al., 2017; Allcott et al., 2018; Grinberg et al.,
2019; Guess et al., 2018; 2019; etc.)
• Data from CrowdTangle: any Facebook posts from public pages / groups / verified profiles that
contained links to any of these domains
• 1 Jan. 2016 to 31 Mar. 2021: 42.6m posts
• Limitations:
• Only public pages / groups / verified profiles (from here: spaces) – no non-public sharing
• CrowdTangle coverage for spaces with small followings is incomplete / inconsistent
• ‘Fake news’ domain lists largely US- / Anglocentric

39. CRICOS No.00213J
US
election
Inauguration
US
election
Capitol
riots
Inauguration
Pandemic
Impeachment
#1

40. CRICOS No.00213J
Approach
• Network analysis:
• Link-sharing network:
• Bipartite network: space ↔ domain
• Structure expected to show clusters commonly sharing same domains
• Computational content analysis of posts per cluster
• Facebook on-sharing network:
• Public Facebook spaces sharing other spaces’ posts that contain FakeNIX links
• Monopartite network: public space ↔ public space
• Structure expected to show common interests, beliefs, politics

41. CRICOS No.00213J
Facebook spaces sharing
links to FakeNIX domains
Link-Sharing Network

42. CRICOS No.00213J
Space-Domain Network
Nodes: public pages, groups, verified profiles / domains in posts
Size: weighted in-degree
Colour: weighted in-degree
FakeNIX domain posts, 1 Jan. 2016 to 31 Mar. 2021

43. CRICOS No.00213J
US progressives
US conservatives
France /
Germany
Italy
Brazil
India
alternative
health
conspiracies
UK
alternative
finance
Space-Domain Network
Nodes: public pages, groups, verified profiles / domains in posts
Size: weighted in-degree
Colour: weighted in-degree
FakeNIX domain posts, 1 Jan. 2016 to 31 Mar. 2021

44. CRICOS No.00213J
Space-Domain Network
Nodes: public pages, groups, verified profiles / domains in posts
Size: weighted in-degree
Colour: weighted in-degree
FakeNIX domain posts, 1 Jan. 2016 to 31 Mar. 2021

45. CRICOS No.00213J
Christians
end times
guns
Tea Party
alternative finance
Detail: US Conservatives

46. CRICOS No.00213J
Nigeria
Pakistan
India
South Africa
Italy
Brazil
Germany
Spain
France
Detail: Language / Country Communities

47. CRICOS No.00213J
climate change denial
antivax
alternative medicine
alternative food
astrology
UFOs
autism
UK left
UK satire
Detail: Conspiracy Theorists

48. CRICOS No.00213J
Facebook spaces sharing
other Facebook spaces’ posts
(that contain links to FakeNIX domains)
Facebook On-Sharing
Network

49. CRICOS No.00213J
Space-Space Network
Nodes: public pages / groups / verified profiles
Size: weighted in-degree
Colour: weighted degree
Posts containing FakeNIX links shared from other Facebook spaces

50. CRICOS No.00213J
Space-Space Network
Posts containing FakeNIX links shared from other Facebook spaces
Spanish
French
Berniecrats
US Progressives
US Conservatives
Indian
Italian
German
climate change
denial
alternative medicine
alternative health
Brazilian

51. CRICOS No.00213J
Detail: Conspiracy Theories

52. CRICOS No.00213J
Detail: RT/Sputnik

53. CRICOS No.00213J
Summary

54. CRICOS No.00213J
Limitations and Outlook
• Limitations:
• public Facebook spaces only (pages, groups, and verified profiles)
• no data on non-public groups
• no data on public or non-public profile activity
• domain-level analysis only
• Outlook:
• general activity of most active FakeNIX sharers (including posts without FakeNIX links)
• further thematic analysis: which topics are prominent in which network contexts
• longitudinal analysis: changes over time (e.g. pre- / post-COVID)
• cross-platform analysis: comparisons with Twitter etc.

55. CRICOS No.00213J
This research is funded by the ARC project DP200101317 Evaluating the Challenge of 'Fake News'
and Other Malinformation.
It is also supported by the Australian Research Council Centre of Excellence for Automated Decision-
Making and Society.
Facebook data are provided courtesy of CrowdTangle.
Acknowledgments

56. CRICOS No.00213J
FakeNIX Sources
Allcott, H., Gentzkow, M., & Yu, C. (2018). Trends in the Diffusion of Misinformation on Social Media. https://arxiv.org/abs/1809.05901v1
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019). Fake News on Twitter during the 2016 U.S. Presidential Election. Science,
363(6425), 374–378. https://doi.org/10.1126/science.aau2706
Guess, A., Nyhan, B., & Reifler, J. (2018). Selective Exposure to Misinformation: Evidence from the Consumption of Fake News during the 2016 US Presidential
Campaign. Dartmouth College. http://www.dartmouth.edu/~nyhan/fake-news-2016.pdf
Guess, A., Nagler, J., & Tucker, J. (2019). Less than You Think: Prevalence and Predictors of Fake News Dissemination on Facebook. Science Advances, 5(1),
eaau4586. https://doi.org/10.1126/sciadv.aau4586
Shao, C., Ciampaglia, G. L., Flammini, A., & Menczer, F. (2016). Hoaxy: A Platform for Tracking Online Misinformation. Proceedings of the 25th International
Conference Companion on World Wide Web, 745–750. https://doi.org/10.1145/2872518.2890098
Starbird, K. (2017, March 15). Information Wars: A Window into the Alternative Media Ecosystem. Medium. https://medium.com/hci-design-at-uw/information-wars-a-
window-into-the-alternative-media-ecosystem-a1347f32fd8f

57. CRICOS No.00213J
Next Time

58. CRICOS No.00213J
Readings
5. 21.10.: Weaponising Newssharing: ‘Fake News’ and Other Malinformation
Bruns, A., Harrington, S., & Hurcombe, E. (2021). Coronavirus Conspiracy Theories: Tracing
Misinformation Trajectories from the Fringes to the Mainstream. In M. Lewis, E. Govender, & K.
Holland (Eds.), Communicating COVID-19: Interdisciplinary Perspectives (pp. 229–249). Springer.
https://doi.org/10.1007/978-3-030-79735-5_12
6. 28.10.: Meet the Audience: How Journalists Adapt to Social Media
Bruns, A. (2018). Meet the Audience: How Journalists Adapt to Social Media. Gatewatching and
News Curation: Journalism, Social Media, and the Public Sphere. Ch. 5. Peter Lang.

59. CRICOS No.00213J
Lecture
• NOTE:
• Week 6 lecture will be pre-recorded and made available through the OLAT site.