Creating Trustworthy AI: A Mozilla White Paper

AI White Paper
Mozilla Foundation
v1.0

Mozilla Conﬁdential
Agenda
2
Challenges
Challenges accelerated or
deepened by AI.
Introduction
Overview
What are the goals of the
white paper?
Questions
Timeline
Timeline for reviewing and
publishing the paper.
Pathway Forward
Unpacking Mozilla’s AI
Theory of Change.
3.
2.
1.
6.
5.
4.

Shifting
industry
norms
Building new
tech and
products
Generating
demand
1
2
3
Creating
regulations
and incentives
4
Agency
All AI is designed with
personal agency in mind.
Privacy, transparency, and
human well-being are key
considerations.
Accountability
Companies are held to account when
their AI systems make discriminatory
decisions, abuse data, or make people
unsafe.
A
B
3
Overview Introduction Challenges Pathway Forward Timeline Questions
AI Theory of Change
Overview

Why a white
paper?
4
■ Charts the provenance of our ideas and
thinking around AI.
■ Deﬁnes Mozilla’s distinct approach to
trustworthy AI.
■ Unpacks Mozilla’s AI theory of change, a
detailed map for our work.
■ Helps us invite others to collaborate and build
oﬀ our work. (Partners see Mozilla as a strong
and trusted partner in the AI space).

Mozilla Conﬁdential 5
Introduction

Why Mozilla?
6
■ Mozilla has a rich history of reimagining
computing norms to favor openness and
innovation.
■ Mozilla has historically been a convener of
disparate groups that point towards a
common goal.
■ We’re at an inﬂection point in the
development of AI that’s not so diﬀerent
from the early web.
■ Many of the challenges posed by AI are not
new. AI adds a new layer of complexity to key
issues Mozilla has already been working on.

Deﬁnitions
7
■ AI: The term AI is vague, but has largely come
to represent a broad assemblage of
technologies and techniques.
■ Trustworthy AI: AI that is demonstrably
worthy of trust. Privacy, transparency and
human well-being are key considerations and
there are mechanisms for accountability.
■ Consumer technology: Products and
services used by or purchased by the broad
public. Note: B2B tech or tech used by
governments/law enforcement would fall
outside of this scope.

● Industry: Current incentives in the tech industry have resulted in business models that
rely on unfettered access to data. The industry is dominated by a handful of tech giants
who wield immense market and political power.
● Regulators: AI development has largely outpaced regulations, resulting in an
environment where ideas are tested and technologies are deployed to millions of people
without proper oversight or transparency.
● Consumers: People feel increasingly powerless. Consumers do not have the information
they need to make educated choices about which products to purchase or which
platforms to use.
The current state
8

Challenges

Challenges
posed by AI
10
1. Monopoly and centralization
2. Data governance and privacy
3. Bias and discrimination
4. Accountability and transparency
5. Industry norms
6. Exploitation of workers and the
environment
7. Safety and security

Companies have a tendency to
stockpile data in order to maintain
their competitive advantage.
Once AI enters the equation,
though, it creates an endless
cycle: Those companies who
dominate the market have greater
access to data, which allows them
to develop better machine learning
models, which enables them to
collect even more data.
1. Monopoly and centralization
11
Only a handful of
tech giants have
the resources to
build AI, stiﬂing
innovation and
competition.
For “platform monopolies” like
Facebook and Google that amass
huge troves of data about how
people behave online, the
competitive advantage is even
more pronounced.
Rapid consolidation of the AI
space is likely to continue, as the
most dominant tech companies
acquire their AI competitors and
the data that comes with them.

Privacy concerns intensify with the
development of AI. Vast amounts of
training data (images, text, video, or
audio) are required to teach
machine learning models how to
recognize patterns and predict
behavior.
Machine learning incentivizes
companies to collect user data
without obtaining meaningful
consent and without suﬃcient
privacy considerations.
2. Data governance and privacy
12
Because AI requires access
to large amounts of
training data, companies
and researchers are
incentivized to develop
invasive techniques for
collecting, storing, and
sharing data without
obtaining meaningful
consent.
As AI continues to drive up the
value of consumer data,
information asymmetry will
continue to increase between
users and the companies
collecting their data.1
1
Ginger Zhe Jin, “Artiﬁcial Intelligence and Consumer
Privacy,” Working Paper (National Bureau of Economic
Research, January 2018),
https://doi.org/10.3386/w24253.

Every dataset comes with its own
set of biases, and it is impossible to
build a fully unbiased AI system.
Often the bias exhibited in an AI
system is the result of incomplete
or biased training data.
Sometimes the bias in an AI system
occurs when the algorithm
unintentionally latches onto the
wrong things in the dataset to
make predictions.
3. Bias and discrimination
13
AI relies on
computational models,
data, and frameworks
that reﬂect existing
bias, often resulting in
biased or
discriminatory
outcomes.
Even when steps have been
taken to reduce bias in a model,
that system can still make
decisions that have a
discriminatory eﬀect.
Computer scientists are rallying
around values like “fairness,
accountability, and transparency”
but this perspective often lacks a
justice or equity perspective.

Many platforms develop closed
algorithms that rapidly generate,
curate, and recommend content.
Platforms are now in a position
where they are making decisions
that will shape society — and there
isn’t adequate oversight.
So-called “black box” algorithms
defy mechanisms for explainability
and accountability, which is
complicated by the fact that many
corporate algorithms remain
trade secrets.
4. Accountability and transparency
14
Companies often
don’t provide
transparency into
how theirAI systems
work, impairing legal
and technical
mechanisms for
corporate
accountability.
Experts have spent years trying
to boost the overall
interpretability and
explainability of AI — whether a
machine learning system can be
understood by and explained to
a human.
Diﬀerent methods of building
AI inherently have diﬀerent
levels of explainability. And
methods for explainability
depend on what kind of
transparency is desired.1
1
Andrew D. Selbst and Solon Barocas, “The Intuitive Appeal of Explainable Machines,” SSRN Scholarly Paper (Rochester, NY: Social Science Research
Network, March 2, 2018), https://doi.org/10.2139/ssrn.3126971.

Market pressures — paired with
weak legal limits — has contributed
to a culture in which new products
are not subjected to critical
examination, suﬃcient testing, or
oversight.
AI is built with a set of assumptions
that have gone unchallenged, and
companies may optimize for a
narrow set of values, such as
proﬁtability, engagement, and
growth.
5. Industry norms
15
Companies are pressured
to build and deployAI
rapidlywithout pausing
to ask critical questions
about the human and
societal impacts. As a
result, AI systems are
embedded with values and
assumptions that are not
questioned in the
development lifecycle.
A real crisis of diversity
(professional, cultural, ethnic,
gender, socioeconomic, and
geographic) contributes to this
problem.
Many engineers, product
managers, designers, and
investors consider
responsibility for AI to be
outside the scope of their job.

AI development has spurred
companies to collect increasingly
large amounts of training data,
resulting in unprecedented levels
of energy consumption and
expanding the need for data
centers, which require space and
enormous amounts of cooling
resources.
There is little to no information
about how much energy big tech’s
algorithms consume, but data
suggest the ad tech industry is
the biggest pollutant in this area.
6. Exploitation of workers and the environment
16
The workers who
perform the invisible
work of maintaining AI
systems are particularly
vulnerable. And, the
climate crisis is being
accelerated byAI, which
intensifies energy
consumption and speeds
up the extraction of
natural resources.
Companies building AI-powered
services rely on a vast, invisible
network of on-demand workers
to clean and label datasets, and
to train and improve models.
There are few employment laws
globally that reflect the realities
of the gig economy. This labor is
often precarious and
temporary, with few benefits
or support.

Algorithmic curation is
increasingly playing a role in
information warfare as
computational propaganda has
become more sophisticated and
subtle. AI can be used to surface
targeted propaganda,
misinformation, and other kinds of
political manipulation.
Algorithmic curation creates
opportunities for a range of actors
to exploit or “game” those systems
for political and/or ﬁnancial gain.
7. Safety and security
17
Malicious actors
may be able to carry
out increasingly
sophisticated
attacks by
exploiting the
vulnerabilities of
intelligent systems.
AI can also be used to automate
labor-intensive cyberattacks like
spear phishing, carry out new
types of attacks like voice
impersonation, and exploit AI’s
vulnerabilities with adversarial
machine learning.1
1
Miles Brundage et al., “The Malicious Use of
Artiﬁcial Intelligence: Forecasting, Prevention,
and Mitigation,” ArXiv:1802.07228 [Cs], February
20, 2018, http://arxiv.org/abs/1802.07228.

Pathway Forward

AI Theory
of Change
Shifting
industry
norms
Building new
tech and
products
Generating
demand
1
2
3
Creating
regulations
and incentives
4
Agency
All AI is designed with
personal agency in mind.
Privacy, transparency, and
human well-being are key
considerations.
Accountability
Companies are held to account when
their AI systems make discriminatory
decisions, abuse data, or make people
unsafe.
A
B
19

1. Shifting industry norms: The people building AI increasingly use trustworthy AI
guidelines and technologies in their work.
2. Building new tech and products: Trustworthy AI products and services are increasingly
embraced by early adopters.
3. Generating demand: Consumers choose trustworthy products when available and
demand them when they aren’t.
4. Creating regulations and incentives: New and existing laws are used to make the AI
ecosystem more trustworthy.
20
AI Theory of Change

AI Theory of Change
SHIFTING
INDUSTRY
NORMS
Best practices emerge in
key areas of trustworthy AI,
driving changes to industry
norms.
Engineers, product managers,
and designers with trustworthy
AI training and experience are in
high demand across industry.
Diverse stakeholders —
including communities and
people historically shut out of
tech — are involved in the
design of AI.
There is increased
investment in and
procurement of trustworthy
AI products, services and
technologies.
BUILDING NEW
TECH &
PRODUCTS
More foundational
trustworthy AI technologies
emerge as building blocks
for developers.
Transparency is included as a
feature in more AI enabled
products, services, and
technologies.
Entrepreneurs develop — and
investors support —
alternative business models
for consumer tech.
The work of artists and
journalists helps people
understand, imagine, and
critique what trustworthy AI
looks like.
GENERATING
DEMAND
Trustworthy AI products
and services emerge that
serve the needs of people
and markets previously
ignored.
Consumers are increasingly
willing and able to choose
products critically based on
information regarding AI
trustworthiness.
Citizens are increasingly
willing and able to pressure
and hold companies
accountable for the
trustworthiness of their AI.
A growing number of civil
society actors are promoting
trustworthy AI as a key part
of their work.
CREATING
REGULATIONS &
INCENTIVES
Governments develop the
vision, skills, and capacities
needed to eﬀectively
regulate AI, relying on both
new and existing laws.
Progress towards trustworthy AI
is made through wider
enforcement of existing rules
like the GDPR.
Regulators have access to the
data and expertise they need
to scrutinize the
trustworthiness of AI in
consumer products and
services.
Governments develop
programs to invest in and
incent trustworthy AI.
21

AI Theory of Change
22
SHIFTING
INDUSTRY
NORMS
norms.
design of AI.
There is increased
investment in and
technologies.
BUILDING NEW
TECH &
PRODUCTS
More foundational
for developers.
technologies.
for consumer tech.
looks like.
GENERATING
DEMAND
ignored.
trustworthiness.
and hold companies
accountable for the
of their work.
CREATING
REGULATIONS &
INCENTIVES
like the GDPR.
to scrutinize the
services.
Governments develop

● Dozens of guidelines for “ethical AI” have been published in recent years.
○ Prominent examples: EU’s High-Level Expert Group, the Partnership on AI, the Organization for
Economic Co-operation and Development (OECD), Google, SAP, the Association of Computing Machinery
(ACM), Access Now
● Frameworks agree on several core principles.
○ The most common principles included were transparency (86.9% of frameworks), justice and fairness
(81.0%), a duty not to commit harm (71.4%), responsibility (71.4%), privacy (56.0%), and human
well-being (48.8%).1
● But there are major differences across sectors about what they mean and how they should be
implemented.
○ In their definitions of transparency, nonprofits and governments refer to audits and oversight, whereas
industry refers to technical solutions to transparency, like explainability.
1.1 Best practices emerge in key areas of trustworthy AI, driving
changes to industry norms.
23
1
Anna Jobin, Marcello Ienca and Effy Vayena, “The global landscape of AI ethics guidelines,” Nature Machine Intelligence, vol. 1, no. 9, Sept. 2019, pp.
389–99, https://www.nature.com/articles/s42256-019-0088-2

● Engineers and other AI domain experts wield a great degree of decision-making power in
development and deployment of AI systems.
● By supporting education and training in building tech responsibly, we aim to put pressure on
companies seeking to attract top engineering talent.
○ The traditional approach to tech ethics education in CS is far removed from the day-to-day
experience of engineers. A skills-based, situated pedagogy gets students one step closer to
operationalizing trustworthy AI principles in the workplace.
● Crucially, research suggests that the actions of internal advocates won’t have impact unless their
work is aligned with organizational practices.1
1.2 Engineers, product managers, and designers with trustworthy
AI training and experience are in high demand across industry.
24
1
Michael Madaio et al., “Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI,” March 19, 2020,
https://www.microsoft.com/en-us/research/publication/co-designing-checklists-to-understand-organizational-challenges-and-opportunities-around-f
airness-in-ai/.

● The diversity crisis in AI has a direct link to problems with bias in AI.
● The teams building AI should strive to reﬂect the diversity of the people who use the technology,
representing a range of identities, communities, and perspectives.
● Diverse communities should be consulted throughout the AI design and development process.
● Companies must foster an open, transparent culture in which the status quo can be questioned
or challenged without fears of retaliation.
○ In its analysis of the diversity crisis in AI, AI Now concluded that a worker-driven movement
aimed at addressing inequities holds the most promise for pushing for real change in diversity.1
1.3 Diverse stakeholders — including communities and people
historically shut out of tech — are involved in the design of AI.
25
1
Sarah Myers West, Meredith Whittaker, and Kate Crawford, “Discriminating Systems: Gender, Race, and Power in AI,” AI Now Institute,
https://ainowinstitute.org/discriminatingsystems.pdf.

● Although there has been a rise in “impact investments” in socially responsible companies and
startups, there is still a lot of work that needs to be done to ensure trustworthy AI products are
getting the funding they need to become viable.
● Tech investors are paying more attention to privacy.
● Tech companies are paying attention to privacy in their acquisition strategy.
● There is a clear opportunity now for such “impact investors” who care about building tech
responsibly to shape the AI product landscape.
1.4 There is increased investment in and procurement of
trustworthy AI products, services and technologies.
26

AI Theory of Change
27
SHIFTING
INDUSTRY
NORMS
norms.
design of AI.
There is increased
investment in and
technologies.
BUILDING NEW
TECH &
PRODUCTS
More foundational
for developers.
technologies.
for consumer tech.
looks like.
GENERATING
DEMAND
ignored.
trustworthiness.
and hold companies
accountable for the
of their work.
CREATING
REGULATIONS &
INCENTIVES
like the GDPR.
to scrutinize the
services.
Governments develop

● A first major step towards better products and services is developing technological building
blocks that can power more responsible AI. These building blocks could include alternative data
governance models, privacy-preserving methods for machine learning, and decentralized, open
source datasets.
● Innovations in privacy-preserving AI include:
○ Edge computing / decentralized computing
○ Federated learning
○ Differential privacy
○ Homomorphic encryption
● Legal innovations in data governance include:
○ Information fiduciaries
○ Data trusts
○ Data co-ops
● And: We need trustworthy pre-trained models & datasets.
2.1 More foundational trustworthy AI technologies emerge as
building blocks for developers.
28

● Tech infrastructure:
○ Explainability: The methods used to explain a particular system depend on what kind of
algorithm or ML technique is being used.
○ Auditability: While developers should be regularly auditing their AI systems, they can also
build those systems in a way that makes them easier to audit by third parties.
○ Human-in-the-loop: Human in the loop means that humans are directly involved in
training, tuning, and verifying the data used in an ML system.
● Product design:
○ User control: Platforms and services can be designed in a way that gives users greater
control and agency over the algorithm’s inputs/outputs.
○ Archives/Libraries: Platforms develop transparency products and oﬀerings. This is part of
a broader bulk disclosure demand.
2.2 Transparency is included as a feature in more AI enabled
products, services, and technologies.
29

● Companies that demonstrate they care about people’s privacy and well-being increasingly have
a market advantage.
● There is a hunger in the market for diﬀerent business models that aren’t focused on
aggressively monetizing people’s data.
● Examples of alternative business models:
○ Set up the platform so that people pay to use it.
○ For two-sided businesses, opt to use privacy-preserving methods of doing data analysis.
Oﬀers a new way to identify patterns without exploiting people’s data.
2.3 Entrepreneurs develop — and investors support —
alternative business models for consumer tech.
30

● Journalists can serve as corporate watchdogs by investigating computational systems, and they
can also help us understand what is happening by providing context and evidence.
● Artists are exposing the limitations and shortcomings of AI.
○ Artists critique current systems and imagine diﬀerent ones by providing us a new lens
through which we can see our world.
○ Art is also a speculative tool that helps us see what alternative worlds and technologies
could look like.
2.4 The work of artists and journalists helps people understand,
imagine, and critique what trustworthy AI looks like.
31

AI Theory of Change
32
SHIFTING
INDUSTRY
NORMS
norms.
design of AI.
There is increased
investment in and
technologies.
BUILDING NEW
TECH &
PRODUCTS
More foundational
for developers.
technologies.
for consumer tech.
looks like.
GENERATING
DEMAND
ignored.
trustworthiness.
and hold companies
accountable for the
of their work.
CREATING
REGULATIONS &
INCENTIVES
like the GDPR.
to scrutinize the
services.
Governments develop

● A new market of privacy-forward consumers
○ A new wave of startups whose core focus is bringing technologies like federated learning
into consumer products.
○ Hints that established big tech players want to tap into the market for privacy.
● People who speak non-dominant languages or who use non-Latin characters have historically
been left out of products.
○ Open source initiatives aimed at inclusion and privacy, e.g. Mozilla’s Common Voice
3.1 Trustworthy AI products and services emerge that serve the
needs of people and markets previously ignored.
33

● At the moment, consumers don’t feel they can make educated choices about what products to
buy or platforms to use.
● As more products using trustworthy AI reach the market, consumers will need better
information about who and what to trust.
○ Mozilla’s Privacy Not Included
○ Consumer Reports’ Digital Standard
○ Data Nutrition Project
3.2 Consumers are increasingly willing and able to choose
products critically based on information regarding AI
trustworthiness.
34

● As we wait for clear consumer protection regulations or a mature market for trustworthy AI
products and services to emerge, consumers will need to pressure companies directly.
● Direct consumer campaigns with precise asks for product changes and transparency is one way
to pressure companies to change their practices.
3.3 Citizens are increasingly willing and able to pressure and hold
companies accountable for the trustworthiness of their AI.
35

● Over the last 25 years, a number of public interest organizations have emerged to promote
digital rights and a healthy internet.
● A new crop of AI-focused public interest organizations has also emerged.
● Established, non-tech organizations are getting involved:
○ Increased focus on privacy, data, and AI in traditional consumer rights groups.
○ Increased interest by civil and human rights organizations in the ways in which AI will impact the
communities they serve.
● Building alliances between digital rights groups and groups from other public interest sectors is
likely the most eﬀective way to meet this need.
3.4 A growing number of civil society actors are promoting
trustworthy AI as a key part of their work.
36

AI Theory of Change
37
SHIFTING
INDUSTRY
NORMS
norms.
design of AI.
There is increased
investment in and
technologies.
BUILDING NEW
TECH &
PRODUCTS
More foundational
for developers.
technologies.
for consumer tech.
looks like.
GENERATING
DEMAND
ignored.
trustworthiness.
and hold companies
accountable for the
of their work.
CREATING
REGULATIONS &
INCENTIVES
like the GDPR.
to scrutinize the
services.
Governments develop

● There’s evidence that policymakers are listening to technologists from civil society. But
nonprofits don’t always have the technical capacity and they are often up against tech lobbyists
and experts representing the interests of big tech companies.
● Policymakers are strengthening their capacity by working with more technologists.
○ Emerging field of “public interest tech” has enabled technologists to influence tech policy.
● Some governments are developing AI-specific centers of expertise.
● Areas to invest:
○ Expanding cross-disciplinary university programs that combine public policy and tech, and
growing the number of research institutions with a focus on AI
○ Creating centers of tech expertise that can be used across departments
4.1 Governments develop the vision, skills, and capacities needed
to effectively regulate AI, relying on both new and existing laws.
38

● Governments are working together to develop global governance frameworks for AI.
○ In 2019, 42 countries took a critical step when they came together to endorse a global
governance framework on AI, the OECD AI Principles.1
The G20 adopted a set of global AI
Principles, largely based on the OECD framework.
● At the same time, countries are putting together their own governance frameworks.
○ European Commission’s 2020 White Paper
○ UK Lords Select Committee’s 2017 AI guidelines
○ China’s 2019 Governance Principles for Responsible AI
○ Singapore’s 2020 Model AI Governance Framework
● The EU’s vision is the most mature. But there’s a gap: The EU has yet to consider the use of AI in
consumer technologies as “high risk”, despite the fact that such technologies pose major
collective risks.
39
4.1 Governments develop the vision, skills, and capacities needed
to eﬀectively regulate AI, relying on both new and existing laws.

● Existing laws and regulations that protect data rights can be wielded in a meaningful way to
address many of the challenges outlined in this paper.
● Existing privacy laws like the GDPR
○ The GDPR has been used to pressure companies into taking data security seriously and to
tackle the surveillance economy and rampant data collection that powers AI.
○ But there are parts of the GDPR that apply to AI that have not yet been tested
■ Article 22: “Automated individual decision-making, including profiling” - mandates that AI cannot
be used to make decisions that have significant impact AND affirms a ‘right to explanation’
■ Article 5: “Principles Relating to Processing of Personal Data” - requires that data processing is fair
AND affirms the principle of data minimization
■ Article 35: “Data Protection Impact Assessment” - requires data protection impact assessments.
4.2 Progress towards trustworthy AI is made through wider
enforcement of existing laws like the GDPR.
40

● Antitrust law
○ Antitrust laws could be applied to break up monopolies in the tech industry, which would
help spur competition and innovation in AI.
○ In the EU, authorities have not shied from imposing ﬁnes on big tech companies based on
competition law.
○ In the U.S., a renewed interest in antitrust laws among legal scholars and regulators has
presented an opportunity to strengthen competition policy.
4.2 Progress towards trustworthy AI is made through wider
enforcement of existing laws like the GDPR.
41

● Full transparency has limitations: it often ignores systems of power, obscures itself further by
overwhelming people, and can promote a false sense of knowledge.1
● Transparency in this context could mean many things:
○ Source code / open source
○ Training data documentation
■ A comprehensive list of all the datasets used, an assessment of the quality of the datasets, an
explanation of how the datasets were manipulated, any records of possible sources of bias, and a
plan for how to account or correct for that bias.
○ AI documentation
■ The model’s training methods, processes and techniques used to test and validate the AI, what
values the model is optimizing for, weights for each parameter at the outset, etc. Should also
include normative explanations for why a particular method was chosen.
○ Data archives/APIs (see 2.2)
4.3 Regulators have access to the data and expertise they need to
scrutinize the trustworthiness of AI in consumer products and
services.
42
1
Mike Ananny and Kate Crawford, “Seeing without Knowing: Limitations of the Transparency Ideal and Its Application to Algorithmic Accountability,”
New Media, Dec 13, 2016, https://doi.org/10.1177/1461444816676645.

● Governments are developing industrial policy that matches their policy goals and vision for AI.
● Governments are developing a procurement strategy that matches their strategic vision for AI.
○ Cities Coalition for Digital Rights
○ UK’s “Guide to using AI in the Public Sector”
● Government agencies adopt procurement guidelines directly into the terms and conditions of
vendor contracts.
4.4 Governments develop programs to invest in and incent
trustworthy AI.
43

Creating Trustworthy AI: A Mozilla White Paper

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Creating Trustworthy AI: A Mozilla White Paper

Similar to Creating Trustworthy AI: A Mozilla White Paper (20)

Recently uploaded

Recently uploaded (20)

Creating Trustworthy AI: A Mozilla White Paper