A talk given at the American Libraries Association Annual Conference on June 24, 2018, New Orleans, LA.

1. A Pedagogical Approach
to Web Scale Discovery
User Interface
Sunday, June 24
9:00 - 10:00 AM
MCC Rm 286-287
Please complete and return the
quick evaluation card to the
volunteers as you leave.
Your feedback is important!
To provide full evaluation
responses of LITA @ 2018 ALA
Annual conference complete this
full survey using any of your
devices:
▪ bit.ly/litaatannual2018

2. A Pedagogical Approach to
Web Scale Discovery
User Interface
#wsd_pedagogy
Bohyun Kim
ChiefTechnologyOfficer &Associate Professor,
University of Rhode Island Libraries
American LibrariesAssociationAnnualConference, June 24, 2018

3. I.WebScale Discovery (WSD)

4. (a) Background: A response to the rapid
increase of library resources on the Web

5. CISCOforecast: http://www.cisco.com/c/m/en_us/solutions/service-provider/vni-forecast-highlights.html
How Fast Is the Internet Growing?

6. https://en.wikipedia.org/wiki/Zettabyte

7. http://www.brookings.edu/~/media/research/files/papers/2015/02/17-future-of-work-in-machine-
age/work_in_machine_age_february_2015_final.pdf

8. Global scientific output doubles every nine years
http://blogs.nature.com/news/2014/05/global-scientific-output-doubles-every-nine-years.html

9. https://creators.vice.com/en_us/article/gyxe4m/refik-anadol-1-7-million-documents-library-of-babel

10. Libraries have …
• Physical holdings
– e.g. books, journals, newspapers, microfiche etc.
• Archival materials, finding aids, other locally-held items
• Licensed e-resources online
– e.g. database (full-text or A&I), e-books, e-journals
• Digital resources and electronic media items locally created
or locally held
• Library discovery beyond one individual library

11. Article-level discovery in a single search box

12. What Is Web Scale Discovery (WSD)?
• “A pre-harvested central index coupled with a richly featured
discovery layer providing a single search across a library’s
local, open access, and subscription collections.”
- Athena Hoeppner, “The Ins andOuts of EvaluatingWeb-Scale Discovery Services,”
Computers in Libraries,April 2012, http://www.infotoday.com/cilmag/apr12/Hoeppner-Web-
Scale-Discovery-Services.shtml.
• A central index of a commercial library WSDS includes over 1
billion records.
• Same idea as the federated search, but faster and more
accurate

13. RecreateGoogle-likeSearchUX
: Page-level Discovery
https://www.google.com/search/howsearchworks/crawling-indexing/

14. http://libguides.luc.edu/ucwr110/searchtools
Complexity of Library Discovery
• physical,
• digital,
• local,
• global (= onWWW)
• free,
• licensed,
• textual,
• non-textual
(AV, images, maps, etc.)
• and more…

15. WSD promises :
• Google-like discovery that is …
• Fast and extensive
• Relevant and seamless
• Search, retrieval, and presentation of the collective results in
multiple sources, types, and formats from within a single
interface
• And the full integration of article-level discovery, access,
and delivery

16. Discovery happens mostly elsewhere.
Cody Hanson et al., “Discoverability Phase 1 Final Report,” (University of Minnesota
Libraries, March 13, 2009), http://conservancy.umn.edu/handle/11299/48258.

17. Let’s Try a Few Searches
a) heart attack
b) diabetes management icu

20. Search Engine Results Pages include:
• Top results
• Encyclopedia-like entry
(from Knowledge Graph)
• Top stories from the news media
• “People also ask” (= FAQs)
• Related searches

22. MeSH Terms: myocardial infarction

27. A Search Query More Specific?
a) heart attack
• The selection criteria of results – Mysterious
b) diabetes management icu

30. Keyword

31. MESH Term & Journal Title

33. Verdict?

34. Will there be a time when library discovery
gets as good as internet search engines?

35. Why Do ISE Results Make More Sense?
https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-10-46

36. Library WSD: Known Weaknesses
• Known item search
• Non-textual material (graphic, video, geospatial, data, etc.)
• Non-English items
• Cross-language search

37. Library WSD: Continuing Challenges
• Varied levels of index coverage (full-text, A&I) and
interoperability with other systems
• Relevancy
• Keeping links up-to-date
• Facets and advanced search features
• More UI improvements based upon the observation of user
behavior

38. TheUX of Library Discovery:
a focus on a more intuitiveUI &Google-likeUX

39. II.Google & Internet Search
Engines

40. What If Library Discovery Becomes As
Good As Google or Other ISEs?

41. Ranking Information (Evaluation) ≠ Neutral
• “One of the reasons this is seen as a neutral process is because
algorithmic, scientific, and mathematical solutions are evaluated
through procedural and mechanistic practices, which in this case
includes tracing hyperlinks among pages. …… while users use the
simplest queries they can in a search box because of the way
interfaces are designed, this does not always reflect how search
terms are mapped against more complex thought patterns and
concepts that users have about a topic. This disjunction between, on
the one hand, users’ queries and their real questions, and, on the
other, information retrieval systems, makes understanding the
complex linkages between the content of the results that appear in a
search and their import as expressions of power and social relations
of critical importance.”
Safiya Umoja Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (New York: NYU Press,
2018), pp.37-38.

42. Biases in Results
• Black girls
• Asian girls
• Unprofessional hairstyles for
work
• Three black boys
• Black-on-white crime
• … and so on
Examples from Safiya Noble, “Toward an Ethic of
Social Justice in Information,” ALA Annual
Conference LITA President’s Program, June 26, 2016.

43. https://www.npr.org/sections/thetwo-way/2017/01/10/508363607/what-happened-when-dylann-roof-asked-google-for-information-about-race

44. ScreenshotbyNPR-https://www.npr.org/sections/thetwo-way/2017/01/10/508363607/what-happened-when-dylann-roof-asked-google-for-
information-about-race
Captured by NPR

45. https://www.npr.org/sections/thetwo-way/2017/01/10/508363607/what-happened-when-dylann-roof-asked-google-for-information-about-race

46. https://searchengineland.com/google-search-quality-crisis-272174

47. Internet Search Engines
• Advertising platforms
• Commodify information including
information about people’s and
groups’ identities
• Ranking is based upon popularity.
• Reinforces existing ideologies,
biases, and prejudices
Screenshot from Danny Sullivan, “A Deep Look at Google’s Biggest-
Ever Search Quality Crisis,” Search Engine Land (blog), April 3, 2017,
https://searchengineland.com/google-search-quality-crisis-272174.

48. Rarely Questioned - ISEs’ Normative Effect
• “Alex Halavais points to the way that heavily used technological
artifacts such as the search engine have become such a normative
part of our experience with digital technology and computers that
they socialize us into believing that these artifacts must therefore
also provide access to credible, accurate information that is
depoliticized and neutral.”
Safiya Umoja Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (New York: NYU Press,
2018), p.25.

49. In reality, ISEs are …
• Interested in search relevance only to the degree in which it can
optimize its ad performance & revenue
• Not a public resource for credible, accurate, and neutral information
• For this reason, public trust in ISE results is deeply problematic.

50. Digital Interfaces Create Material Reality
• “Search does not merely present pages but structures knowledge,
and the results retrieved in a commercial search engine create their
own particular material reality. Ranking is itself information that
also reflects the political, social, and cultural values of the society
that search engine companies operate within, a notion that is
often obscured in traditional information science studies.”
• The opacity of ISEs impedes informed public discourse and harms
civic education by presenting the world in a certain way while
masking that process.
Safiya Umoja Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (New York: NYU
Press, 2018), p.148.

51. What do libraries want to deliver that
Google or other ISEs do not and will not?

52. The Goal of “Library” Discovery
• Library discovery and ISE discovery do not share the same goal.
• [X] ISE-like discovery experience
• [X] To produce a searcher conversant to one UI
• [O] Support and facilitate informed public discourse and civic
education by enabling people to develop an understanding of and
practical skills for critical information evaluation.

53. III.Critical Pedagogy and Library
Discovery

54. The Way We Treat Library Discovery Shows
Our Assumptions …
• Information as a thing
• An information user as a passive consumer
• Discovery reduced to a transactional step and one-time event

55. The “Banking” Concept of Education
• “Education thus becomes an act of depositing, in which the students
are the depositories and the teacher is the depositor. Instead of
communicating, the teacher issues communiques and makes
deposits which the students patiently receive, memorize, and repeat.
This is the “banking” concept of education, in which the scope of
action allowed to the students extends only as far as receiving, filling,
and storing the deposits.”
- Paulo Freire, Pedagogy of the Oppressed, trans. Myra Bergman Ramos (Continuum, 1993), p.53.

56. Alternative: Problem-Posing Education
• “Banking education attempts, by mythicizing reality, to conceal
certain facts which explain the way human beings exist in the world;
problem-posing education sets itself the task of demythologizing.
Banking education resists dialogue; problem-posing education
regards dialogue as indispensable to the act of cognition which
unveils reality. Banking education treats students as objects of
assistance; problem-posing education makes them critical thinkers.”
• Liberates from the influence of false / malicious info, biases, and
prejudices presented as something natural/neutral in ISEs
- Paulo Freire, Pedagogy of the Oppressed, trans. Myra Bergman Ramos (Continuum, 1993), p.64.

57. Commodification of Information in ISEs &
Neoliberalism
• An ideology and belief that the market should govern all dimensions
of human life beyond economy.
• Education = Pursued as a private good to meet the corporate
demand that it provides the skills, knowledge, and credentials for a
job
• “Neoliberalism wages an incessant attack on democracy, public
goods, and non-commodified values. Under neoliberalism everything
either is for sale or is plundered for profit.”
- Henry A. Giroux, “The Terror of Neoliberalism: Rethinking the Significance of Cultural Politics,” College
Literature 32, no. 1 (2005): 1–19, https://doi.org/10.1353/lit.2005.0006.

58. An Alternative Ideal in Critical Pedagogy
• “Consequently, there is little interest in understanding the
pedagogical foundation of higher education as a deeply civic,
political, and moral practice – that is, pedagogy as a practice for
freedom. As schooling is increasingly subordinated to a corporate
order, any vestige of critical education is replaced by training and the
promise of economic security.”
Henry A. Giroux, “Rethinking Education as the Practice of Freedom: Paulo Freire and the Promise of Critical
Pedagogy,” Policy Futures in Education 8, no. 6 (December 1, 2010): 715–21,
https://doi.org/10.2304/pfie.2010.8.6.715.

59. Civic Education
• “… questions of civic education and critical pedagogy (learning how
to become a skilled citizen) are central to the struggle over political
agency and democracy. In this instance, critical pedagogy
emphasizes critical reflexivity, bridging the gap between learning
and everyday life, understanding the connection between power and
knowledge, and extending democratic rights and identities by
using the resources of history.”
Henry A. Giroux, “Critical Pedagogy and the Postmodern/Modern Divide: Towards a Pedagogy of
Democratization,” Teacher Education Quarterly 31, no. 1 (Winter 2004): 31–47.

60. “Library” Discovery Revisited
• [O] An information user as an active civic, political, and moral agent
• [X] Not a passive consumer
• [O] Information as a catalyst for civic, political, and moral practice
• [X] Not a thing

61. When Library Discovery Is Reconceptualized
as a Pedagogical Tool…
• Discovery is no longer reduced to a transactional step and an one-
time event.
• Discovery is envisioned as a way to appropriately present
information to information users, so that they can learn to practice
their civic, political, and moral agency to its fullest potential.
• New Q: Can a library discovery solution be a means to actively
prompt and facilitate critical thinking at the point of search? How
would the UI of such a web scale discovery solution look like?

62. Critical Thinking in Information Literacy
• “… librarians in the academy need to define information itself as the
product of socially negotiated epistemological processes and the raw
material for the further making of new knowledge. … Ultimately,
critical information literacy involves developing a critical
consciousness about information, learning to ask questions about
the library’s (and the academy’s) role in structuring and presenting a
single, knowable reality.”
• The goal of library discovery as learning to question (not obscure)
how information is selected and presented
James Elmborg, “Critical Information Literacy: Implications for Instructional Practice,” The Journal of
Academic Librarianship 32, no. 2 (March 1, 2006): 192–99, https://doi.org/10.1016/j.acalib.2005.12.004.

63. Safiya Umoja Noble, Algorithms of Oppression: How Search Engines Reinforce Racism (New York: NYU Press, 2018), p.180.

64. Contextualization and More
• Contextualize to de-naturalize
• Reframe / re-contextualize inquiries and information
• Prompt critical thinking to lead people to deliberation/action, an
exercise of their freedom as a fully competent civic, political, and
moral agent.

65. ACRL Framework for Information Literacy
• Authority Is Constructed and Contextual
• Information Creation as a Process
• Information Has Value
• Research as Inquiry
• Scholarship as Conversation
• Searching as Strategic Exploration
http://www.ala.org/acrl/standards/ilframework

66. https://lib.mit.edu/search/bento?q=hantavirus

67. TimelineInformation Creation as a Process

68. https://yewno.com/edu/concept/102e6bf8445dff7c922e1cc4e997ebf3
Network Graphs
Research as Inquiry

69. https://scholars.cornell.edu/vis/author-network/igi2
Author Networks
Scholarship as Conversation

70. Presenting Information for Critical Thinking
• As a whole instead of a fragment
• With as much context as possible to emphasize its connection to
other information along with its historic background
• Its connection to political, social, and economic issues that impact
information users should be highlighted, so that the information
provided can support the user’s exercising her/his civic, political,
moral agency.

71. Challenges
• How to achieve this at a scale without requiring a lot of human work
• How to present a large amount of information in a manageable,
understandable, and easily evaluable way

72. Skepticism about Using UI for Pedagogy
• Isn’t prompting and facilitating critical thinking something that
needs in-person education?
• “I'm not sure that a discovery layer can actually do this. By moving
toward a Google-like one-stop shop, we've dumped too much
information on researchers and erased the lines between kinds of
information, which makes it even harder for novice researchers who
tend to see anything they find on the web as a website - whether it's an
e-book, a news article, a scholarly article, or a government report.
When a researcher is presented with what is too much information,
there is little space for deeper evaluation, reflection, and
understanding.” -- from one of my colleagues

73. Need for the UI that Embodies Critical
Pedagogy
• Many don’t have access to a teacher/librarian…
• How to bring critical thinking at the point of search?
• A WSDS that embodies critical pedagogy will deliver search results in
a very different way than Google or other ISEs.

74. With an Eye towards the Future
• Usage-based recommendations
• Open source centralized index
• Open access
• Research data
• LOD
• NLP
• AI

75. “Library” Discovery
• More about people than resources
• A mechanism that counters faulty reasoning, hasty generalizations,
and cognitive biases
• A tool that facilitates informed public discourse and supports civic
education

76. ThankYou!
BohyunKim
@bohyunkim[Twitter]
http://bohyunkim.net/blog[Blog]
ChiefTechnologyOfficer
Universityof RhodeIslandLibraries