Diachronic Analysis of Language Change and Concept Creep

Diachronic Analysis of Language Change
Ekaterina Vylomova
Vylomova, Ekaterina Diachronic Analysis of Language Change 1 / 63

Language changes in many ways!
Phonologically: The First Germanic Sound Shift
bh > b > p > Φ
dh > d > t > Θ
gh > g > k > x

Grammaticalization: Synchronic variation and diachronic change
Old English “willan” (to want) → Modern English auxiliary “will”

Language, 1933: Semantically:
Narrowing as in Old English “mete” (food) > "meat" (edible ﬂesh)
Widening as in Middle English "bridde" (young birdling) > "bird"
Metaphor: Germanic "biting" > "bitter" (harsh of taste)
Metonymy when a thing is referred by the name of something closely associated with it:
Old French "joue" (cheek) > "jaw"

Language, 1933: Semantically:
Synecdoche when the meaning are related as part and whole: Proto-Germanic "*t¯uną"
(fence) > "town"
Hyperbole, i.e. from weaker to stronger meaning: Pre-French "*extonare" (to strike with
thunder) > "astonish"
Litotes, i.e. from stronger to weaker meaning: pre-English "*kwalljan" (to torment) > Old
English "cwellan" (to kill)
Degeneration: Old English "cnafa" (boy, servant) > "knave"
Elevation: Old English "cnight"(boy, servant) > "knight"

What causes language to change?
Social evolution and language change:
Interpersonal communication → the eﬃciency of communication serves as a driving force in
such processes
Sociolinguistic research suggests that much of language change and variation can be attributed
to social structure
Kutuzov (2018) proposes that causes of the processes leading to semantic shifts vary and
comprise linguistic, social, sociolinguistic, cultural, and psychological

Two types of Change
Cultural shift and Linguistic Drift

Two types of Change
Cultural shift and Linguistic Drift
Slow and regular changes Only aﬀect the word’s nearest neighbors!

Cultural Shifts
Culturomics (Mitchel et al., 2011) – the study of cultural and historical changes using
large corpora

Concept Creep (Haslam, 2016)
Harm-related concepts become broader and milder over time

Concept Creep (Haslam, 2016)
Vertical extension: a concept’s meaning becomes less stringent, extending to quantitatively
milder variants of the phenomenon
Horizontal expansion: a concept extends to a qualitatively new class of phenomena

Concept Creep. Pinker, 2011: Rate of deaths in genocides, 1900-2008

Concept Creep. Pinker, 2011: Uxoricide & Mariticide, 1976-2005

Concept Creep. Pinker, 2011: Child Abuse, 1990-2006

Concept Creep. Addiction
Physiological dependency on an ingested substance → psychological compulsion to engage in
non-ingestive behaviors such as gambling or shopping

Concept Creep. Bullying
Peer aggression between children that was repeated, intentional, and perpetrated→ adult
workplace, relaxing the repetition, intentionality, and power imbalance criteria

Concept Creep. Harassment
Inappropriate sexual approaches → nonsexual forms of unwanted attention

Concept Creep. Prejudice
Overt animosity towards ethnic or racial outgroups → non-racial groups, allowing for covert or
non-conscious prejudice

Concept Creep. Trauma
Life-threatening events that are outside the realm of normal experience → vicarious or indirect
experiences of stressful events, including those that are relatively prevalent

Concept Creep. Corpus of Psychology Journals (Vylomova et al., 2019)
Timeline: 1930 – 2017
PubMed
E-Research
871, 340 abstracts from 875 journals resulting in 133, 082, 240 tokens
0
20000
40000
1950 1975 2000

Concept Creep. Concept Frequency Analysis
Unigram frequency distribution over time: a “moving average” smoothing with window size of
1, i.e. f1972 = (f1971 + f1972 + f1973)/3.
0.00
0.02
0.04
1970 1980 1990 2000 2010 2020
harassment trauma addiction bullying prejudice

Concept Creep. Vector-Space Methods
LSA-based
Neural

Concept Creep. Vector-Space Methods
LSA-based
Neural
Also: recall research on various types of bias in that WEs: they may reﬂect many stereotypes

Concept Creep. LSA-based (Sagi et al., 2009)
Step 1 (embeddings over all periods):
40,000 most frequent terms
TF-IDF matrix with logarithmic smoothing
factorize by SVD to 200 dimensions
Step 2 (diachronic embeddings (1980-2017)):
sample 50 sentential occurrences for each period T
extract contextual words within window size = 7
Average embeddings (BoW)
Repeat 10 times

Concept Creep. Semantic Breadth as Cosine Similarity
0.10
0.15
0.20
198x 199x 200x 201x
harassment trauma addiction bullying prejudice
Average cosine similarities over ﬁve decades.

Concept Creep. Diachronic Embeddings
Train separate epoch-speciﬁc models and align
Train word2vec models for each decade
Project embeddings into shared space (e.g., using Procrustes)

Sequentially train for each epoch
Train embeddings for epoch t
Initialize embeddigns for epoch t + 1 with embeddings for epoch t and train

Pre-train globally and then train pre epoch
Pre-train on the whole corpus
Initialize epoch-speciﬁc embeddings with global pre-trained embeddings and train for each
epoch

Concept Creep. Used in the paper: Diachronic embeddings from Hamilton et
al.(2016)
train word2vec CBoW for each decade:
J =
1
T
T
i=1
log
exp wi
j∈[−c,+c],j=0
˜wi+j
V
k=1 exp wk
j∈[−c,+c],j=0
˜wi+j
align embeddings using orthogonal Procrustes: argminθT θ=I θW t − W t+1
F

Concept Creep. Semantic Displacement: cosine distances between decades
cos-dist(wt
, wt+δ
)
1980s-90s 1990s-00s 2000s-10s
addiction
bullying
harassment
prejudice
trauma
0.35 0.23 0.23
0.64 0.27 0.19
0.29 0.21 0.18
0.31 0.26 0.16
0.31 0.18 0.09

Concept Creep. Pair-wise similarity time-series: Addiction
s(t)
(wi, wj) = cos-sim(wt
i , wt
j )
1980s 1990s 2000s 2010s
alcohol
cigarette
drug
food
gaming
heroin
internet
marijuana
narcotic
nicotine
opiate
sex
sexual
smartphone
0.31 0.31 0.3 0.21
0.2 0.11 0.09 0.15
0.4 0.3 0.34 0.28
0.01 -0.02 0.01 0.06
0.04 0.18 0.18 0.39
0.47 0.35 0.33 0.21
0 0.15 0.22 0.33
0.37 0.26 0.18 0.17
0.44 0.31 0.4 0.26
0.15 0.28 0.23 0.21
0.39 0.27 0.38 0.29
-0.09 0.05 0.03 0.04
0.01 0.17 0.15 0.12
0 0 -0.07 0.21

Concept Creep. Pair-wise similarity time-series: Bullying
s(t)
i , wt
j )
1980s 1990s 2000s 2010s
child
peer
physical
relationship
school
verbal
victim
workplace
0.13 0.2 0.2 0.13
0.18 0.31 0.41 0.43
-0.01 0.12 0.19 0.17
-0.02 0.12 0.13 0.12
0.16 0.34 0.34 0.35
-0.02 0.03 0.08 0.09
0.18 0.34 0.49 0.46
0.09 0.26 0.39 0.4

Concept Creep. Pair-wise similarity time-series: Harassment
s(t)
i , wt
j )
1980s 1990s 2000s 2010s
cyber
ethnic
gender
online
peer
physical
racial
school
sexual
student
verbal
woman
workplace
0 -0.01 0.3 0.43
0.07 0.18 0.16 0.17
0.1 0.21 0.2 0.2
-0.13 0.1 0.18 0.25
0.01 0.12 0.21 0.26
0 0.08 0.12 0.15
0.18 0.25 0.32 0.31
0.09 0.05 0.09 0.14
0.18 0.16 0.15 0.25
0.12 0.18 0.19 0.18
-0.12 -0.01 0 0.04
0.2 0.24 0.22 0.2
0.21 0.45 0.41 0.39

Concept Creep. Pair-wise similarity time-series: Prejudice
s(t)
i , wt
j )
1980s 1990s 2000s 2010s
black
discrimination
ethnic
gay
positive
racial
sex
sexual
social
woman
0.42 0.34 0.35 0.33
0.14 0.3 0.32 0.44
0.41 0.44 0.38 0.4
0.28 0.31 0.27 0.36
0.11 0.12 0.19 0.22
0.48 0.5 0.52 0.53
0.24 0.22 0.18 0.12
0.2 0.17 0.17 0.17
0.29 0.26 0.28 0.23
0.2 0.15 0.11 0.13

Concept Creep. Pair-wise similarity time-series: Trauma
s(t)
i , wt
j )
1980s 1990s 2000s 2010s
child
childhood
emotional
interpersonal
physical
psychological
sexual
stress
0.04 0.05 0.04 0.03
0.37 0.36 0.31 0.28
0.22 0.19 0.26 0.22
-0.06 0.07 0.21 0.22
0.19 0.15 0.09 0.03
0.19 0.25 0.31 0.28
0.11 0.2 0.24 0.19
0.29 0.31 0.34 0.4

Concept Creep. Conclusions
Since the 1990s Addiction, Bullying, Harassment have broadened, as the theory of concept
creep would suggest, but the breadth of Trauma has been relatively static and Prejudice
has somewhat narrowed
The analysis of pairwise similarities demonstrated changing patterns of co-occurrence for
each concept that clariﬁed how its meanings have shifted and expanded over four decades
Some concepts have acquired entirely new associations (e.g., cyber-harassment), some
have added new semantic domains (e.g., Addiction incorporating non-ingestive behaviors
such as gaming and smartphone use), and others have shifted emphasis (e.g.,Trauma
becoming associated less with physical injury and more with psychological stress)

Dehumanization
Dehumanization: the act of perceiving or treating people as less
than human
negative evaluations of a target group
denial of agency
moral disgust
likening members of a target group to non-human entities (such as vermin;via metaphors)

Dehumanization
A computational linguistic framework for analyzing dehumanizing
language,with a focus on lexical signals of dehumanization
negative evaluations of a target group
denial of agency
moral disgust
likening members of a target group to non-human entities (such as vermin;via metaphors)

Dehumanization
Initial Dictionary (target terms)

Dehumanization
Data
“New York Times”: from 1986 – 2015, collected by Fast and Horvitz (2016)
Extracted paragraphs containing any word from the predetermined list (+American(s) as a
control variable) → LGBTQ corpus
The LGBTQ corpus consists of 93,977 paragraphs and 7.36 million tokens

Dehumanization
Word Embeddings
Contain biases but might be useful to study social stereotypes (Garg et al., 2018)
Used Word2vec skip-gram (100 dimensions): 1) Train on the whole corpus; 2) use the
resulting vectors to initialize word2vec models for each year of data; 3) zero-center and
normalize all embeddings to alleviate the hubness problem (Dinu et al., 2015)

Dehumanization
WEs: nearest neighbors to average over LGBTQ term vectors

Dehumanization
WEs: nearest neighbors to gay vs. homosexual

Dehumanization
Negative evaluations of a target group: valence
Valence: negative/unpleasant to positive/pleasant
Data: the NRC VAD lexicon (valence: 0(neg.;toxic, nightmare) – 1(pos.; love, happy)) for
20,000 English words
Paragraph-level scores: the average valence score over all words (from NRC VAD) in the
paragraph
Biases in NRC VAD: transsexual(0.264),homosexual(0.333), esbian(0.385),gay(0.388) vs.
heterosexual(0.561),person(0.646),human(0.767),man(0.688), woman(0.865) (these terms
were excluded)

Dehumanization

Dehumanization
Homosexual more negative than gay
LGBTQ groups have become increasingly positively evaluated

Dehumanization
Negative evaluations of a target group: connotation frames
Understand the sentiment directed towards the target group: “A violently attacked B".
Assign subject “A”, the attacker, as -0.6 (strongly negative) and the object “B” as 0.23
(weakly positive).
Extract SVO tuples (using dependency parser); apply connotation frames
Data: Connotation Frames from (Rashkin et al., 2016); 900 English verbs

Dehumanization
Negative evaluations of a target group: connotation frames

Dehumanization
Negative evaluations of a target group: word embeddings
Average valence scores of the 1000 nearest neighbors to the vector representations of
gay,homosexual, all LGBTQ terms, and American for each year

Dehumanization
Negative evaluations of a target group: word embeddings

Dehumanization
Denial of agency
Denial of agency refers to the lack of attributing a target group member with the ability to
control their own actions or decisions (Tipler and Ruscher, 2014).
Connotation frames of agency
Word embedding neighbor dominance

Dehumanization
Denial of agency: connotation frames
Let’s quantify the amount of agency attributed to a target group
"X searched for Y"and "X waited for Y": the verb "searched" gives X high agency and
"waited" gives X low agency (binary)
Data: Connotation Frames for agency (Sap et al., 2017); 2000 transitive and intransitive
verbs
Extract head verbs and their corr. subject NP’s containing any LGBTQ terms

Dehumanization

Dehumanization
LGBTQ groups experience greater denial of agency than "American"
Denial of agency increased over time for all LGBTQ groups

Dehumanization
Denial of agency: dominance
Quantify the amount of dominance: dominance lexicon primarily captures power, which is
distinct from but closely related to agency
Data: NRC VAD lexicon’s dominance annotations (Mohammad et al., 2018), 0–1, 20,000
English words
Highest dominance words: powerful,leadership,success, and govern; Lowest dominance
words: weak,frail,empty, and penniless.
Calculate the average dominance score of the 1000 nearest neighbors to each group label
vector representation

Dehumanization
Denial of agency: dominance

Dehumanization
Moral disgust
Vector similarity to “disgust”
Moral Foundations theory, which postulates that there are ﬁve dimensions of moral
intuitions: care, fairness/proportionality, loyalty/in-group, authority/respect, and
sanctity/purity(Haidt and Graham, 2007).
Moral Disgust: the negative end of the sanctity/purity (dictionary from Graham et al.,
2009: disgust*,sin,pervert)
“Disgust”:average of the term from the “Moral Disgust” dictionary, weighted by word
frequency
calculate the cosine similarity between the group label’s vector and the Moral Disgust
concept vector

Dehumanization
Moral Disgust

Dehumanization
Moral Disgust
All LGBTQ group labels are more closely associated with Moral Disgust than "American"
These associations weaken over time which suggests increased humanization

Dehumanization
Vermin as a Dehumanizing Metaphor
"Vermin" concept: the average of the following vectors, weighted by
frequency:vermin,rodent(s),rat(s) mice,cockroaches,termite(s),bedbug(s),ﬂeas
Calculate cosine similarity between each social group label and the Vermin concept vector

Dehumanization
Moral Disgust

Dehumanization
Conclusions
Increasingly humanizing descriptions of LGBTQ people over time & LGBTQ people have
become more associated with positive emotional language
The labels “gay” and “homosexual” exhibit strikingly diﬀerent patterns
Future: contextualized word embeddings

Diachronic Language Change: Further reading
Some nice overview papers
Diachronic word embeddings and semantic shifts: a survey
Survey of Computational Approaches to Lexical Semantic Change

Diachronic Analysis of Language Change and Concept Creep

Recommandé

Recommandé

Contenu connexe

Similaire à Diachronic Analysis of Language Change and Concept Creep

Similaire à Diachronic Analysis of Language Change and Concept Creep (17)

Plus de Katerina Vylomova

Plus de Katerina Vylomova (16)

Dernier

Dernier (20)

Diachronic Analysis of Language Change and Concept Creep