-

1. What is AI Literacy?
Competencies and Design
Consideration
Hyunwook Lee
2020. 07. 31
CHI 2020
Duri Long, Brian Magerko

2. Contents
•Overview of the Paper
•Defining AI Literacy
•Competencies and Design Consideration
•Conclusion & Future Work
•Summary

3. 2
Overview of the Paper
• Motivation
 AI is increasingly integrated in user-facing technology
• But, most of the public doesn’t understand the concept of AI – even they doesn’t know they use
AI technology [10, 54, 55]  limit the ability to use, collaborate with AI [57]
 HCI community began to address these misconceptions by
• Investigating how people make sense of AI [46], design understandable technology [67]
• investigating of the competencies for communication, working, living with AI is limited
 Enormous attention in AI Education
• Some Company(AI4ALL, ReadyAI) serve the AI Education for public
• Researchers are exploring how to engage children in creative programming activities involving AI
• Educators published guides on how to incorporate AI into K-12(유치원-고등학교)
• Five “big ideas”: 1) Computers perceive the world using sensors 2) Agents maintain
models/representations of the world and use them for reasoning 3) Computer can learn from data
4) Making agents interact with humans is a common challenge for AI developers 5) AI app can impact
society in both positive/negative ways

4. 3
Overview of the Paper
• Main Contribution
 A concrete definition of AI literacy
 Related set of competencies and design Consideration
• Conceptual framework composed of five different themes
 What is AI?
 What can AI do?
 How does AI work?
 How should AI be used?
 How do people perceive AI?

5. 4
Defining AI Literacy
• There are Many different literacy among areas
 Digital Literacy
 Competencies needed to use computational devices [14]
 Computational Literacy
 Ability to use code to express, explore, and communicate idea [40]
 Scientific Literacy
 an appreciation of the nature, aims, and general limitations of science, coupled
with some understanding of the more important scientific ideas [86]
 Data Literacy
 the ability to read, work with, analyze, and argue with data as part of a broader
process of inquiry into the world [36]

6. 5
Defining AI Literacy
• AI Literacy  A set of competencies that enables individuals to …
 Critically evaluate AI technologies
 Communicate and collaborate effectively with AI
 Use AI as a tool online, at home, and in the workspace
• Digital literacy is a prerequisite for AI literacy
• For the full understand of AI, there are overlapped competencies with
data literacy

7. 6
Competencies: What Is AI?
• Even for the experts, definition of AI can be confusing
• Misconceptions about AI
 AI is synonymous with robotics
 Artifacts do not achieve human-level intelligence is not AI
• Competency 1: Recognizing AI
 def. Distinguish between technological artifacts that use and do not use AI
• Discussion on definition of AI
 Nilsson [100] defines AI as “activity devoted to making machines intelligent”
• “quality that enables an entity to function appropriately and with foresight in its environment”
 Schank [116] notes that definitions of intelligence is differ depending on researcher
• AI research has two main goal – “Make intelligent model”, “find out the nature of intelligence”
• Set of traits that comprise general intelligence
• Communication, world knowledge, internal knowledge, intentionality, and creativity
• Ability to learn is most critical part of intelligence
 Brooks and others also suggest synthesized perspectives on intelligence

8. 7
Competencies: What Is AI?
• Competency 2: Understanding Intelligence
 def. Critically analyze and discuss features that make an entity “intelligent”,
including differences between human, animal, and machine intelligence
 Activity like comparing AI devices and AI vs. human can helpful for this competency
• Competency 3: Interdisciplinarity
 def. Recognize that there are many ways to think about and develop “intelligent”
machines. Identify a variety of technologies that use AI, including technology
spanning cognitive systems, robotics, and ML.
• General AI(operating like human) is not achieved  only exists narrow AI
among area
• Competency 4: General vs. Narrow
 def. Distinguish between general and narrow AI

9. 8
Competencies: What AI Can Do?
• Survey in ARM,PEGA indicate that people’s trust in AI is heavily task-
dependent  doesn’t know about AI strength/weakness
 AI is good at detecting patterns from large data, doing repetitive task, making
decisions in controlled environments
 Humans are better at most tasks requiring creativity, emotion, knowledge transfer,
and social interaction
• Competency 5: AI’s Strengths & Weakness
 def. Identify problem types that AI excels at and problems that are more challenging
for AI. Use this information to determine when it is appropriate to use AI and when
to leverage human skills
• Competency 6: Imagine Future AI
 def. Imagine possible future applications of AI and consider the effects of such
applications on the world.

10. 9
Competencies: How does AI work?
• Many people self-reported that they know little about AI
 But these people often develop “folk theories” to explain AI  better understanding of
how AI works can help people to form more accurate mental models
• Cognitive Systems
 Used in a variety of application(e.g. WordNet, IBM’s Watson)
 To understand cognitive system, firstly people should know how computer understand
the world with knowledge representation(Competency 7: representations)
 Cognitive systems use many strategies for decision making and it is different from
human one  high-level understand of the computer decision making
needed(Competency 8: Decision making)
 To help learners to understand decision making procedure, strategies such as
interactive demonstration/visualization, simulation to test hypothesis, and
explanation using storytelling(Design Consideration 1: Explainability)

11. 10
How does AI work?: Machine Learning
• To understand ML, people should know steps involved in ML
 Competency 9(ML Steps)
• Common misconception in university ML course
 Computers think like humans  students want to make connections between
human theories of cognition and machine learning (supports Competency 2)
 ML is fully-automated and doesn’t need human decision-making
 Competency 10(Human Role in AI)
 Students often have difficulty to identify the limit of ML and constraints that may
make ML unsuitable(supports Competency 5)
 According to previous researches, engaging in embodied interaction is helpful to
overcome these misconceptions  Design Consideration 2(Embodied Interaction)

12. 11
Competencies: How does AI work?
• In machine learning, understanding about data is one of the most
important part
 Data Literacy is a prerequisite for AI Literacy  Competency 11(Data Literacy)
 Competency 12(Learning from data)
 Learner should know that data cannot be taken at face-value and require
interpretation  Competency 13(Critically Interpreting Data)
 For the better understand, D’Ignazio and Sulmont et al. encourage educators to
carefully select the datasets they use for lecture, considering the learning steps
 Also D’Ignazio suggests to write data biographies to show the limitation and origins
of data  Design Consideration 3(Contextualizing data)
• Robotics
 To understand robotics, people should know that some AI systems have ability to
physically act on the world(Competency 14: Action & Reaction), and how they
gather data and interface with world(Competency 15: Sensor)

13. 12
Competencies: How Should AI Be Used?
• AI application can impact society both positive and negative ways 
should know key ethical issues surrounding AI(Competency 16: Ethics)
• Key ethical issues surrounding AI
 Privacy/surveillance
 Employment
 Misinformation
 Singularity/concern about harm to people
 Ethical decision making
 Diversity
 Bias/fairness
 Transparency
 Accountability

14. 13
How Do People Perceive AI?: Interpreting AI Systems
• Human tend to understand the actions of agents using theory of mind
 This approach is not a reliable way of making sense of AI, due to the differences
between AI and Human reasoning
• Wardrip-Fruin describes three effects that can arise in the relationship
between the appearance of a digital system and its internal operation
 Eliza effect
• a system uses simple techniques but produces effects that appear complex
 Tale-Spin effect
• a system that has complex internal operations, but externally appears “significantly less complex”
 SimCity effect
• a system that, through play, brings the player to an accurate understanding of the system’s
internal operations

15. 14
How Do People Perceive AI?: Interpreting AI Systems
• To prevent these misunderstanding caused by black-box algorithm, two
design consideration suggested
• Design Consideration 4: Promote Transparency
 def. Promote transparency in all aspects of AI design (i.e. eliminating black-boxed
functionality, sharing creator intentions and funding/data sources, etc.).
 This may involve improving documentation, incorporating explainable AI (Design
Consideration 1), contextualizing data (Design Consideration 3), and incorporating
design features such as interpretative affordances or the Sim-City Effect.
• Design Consideration 5: Unveil Gradually
 To prevent cognitive overload, consider giving users the option to inspect and learn
about different system components; explaining only a few components at once; or
introducing scaffolding that fades as the user learns more about the system’s
operations.

16. 15
How Do People Perceive AI?: Children’s Perceptions of AI
• Children has not developed theory of mind, so their acceptance of the AI
have researched(with AIBO, My Friend Kayla)
• They focus on observable characteristics(e.g. success)
rather than unobservable one(e.g. strategy)
• Age play important role in AI perception
 Over age 8: tend to agree with parent’s assessments
 Younger than 8: overestimate intelligence, often perceiving
agents to be smarter than themselves
• For the AI education in childhood, research shows that
understanding that agent is programmable is the most
important part.(Competency 17)

17. 16
How Do People Perceive AI?: Children’s Perceptions of AI
• After they realize that agent are programmable, there should be some
tools for the program(e.g. Cognimates, eCraft2Learn)
 Design Consideration 6: Opportunities to Program
 Also, designing of the tool should consider the children’s age, because their
perception varying among age  Design consideration 7(Milestone)
• Design Consideration 8: Critical Thinking
• Design Consideration 9: Identity, Values, & Backgrounds
• Design Consideration 10: Support for Parents
• Design Consideration 11: Social Interaction
• Design Consideration 12: Leverage Learners’ Interests

18. 17
How Do People Perceive AI?: Perceptions of AI in Media
• A meta-analysis of NYT and other articles has shown numerous
trends in AI-related coverage
• Another meta-analysis found that recent news about AI in the UK
is heavily dominated(60%) by industry
 12% of article mentioning Elon Musk  consider the
politicized/sensationalized preconceptions about AI(Design
consideration 13)
• In visual media, most of the AI appears to have human-level
intelligence  should introduce other kinds of AI in learning
intervention(Design consideration 14)

19. 18
How Do People Perceive AI?: Perceptions about Learning AI
• In ML courses, some preconceptions often hold:
 Believing ML is important, particularly for the job market
 Hearing of ML through popular, often sensationalized, media
Believing that implementing ML is not accessible without
having a background in CS/Math
 Lowering barriers to entry in AI education is
important(Design Consideration 15)

20. 19
Criticisms
• Design Considerations are useful for the learner-based AI
implementation
• Few of the competency didn’t seem essential for AI learner(e.g.
Imagine Future AI)
• Bias in “How does AI work?”
 Machine Learning have much more than Robotics