SlideShare une entreprise Scribd logo
1  sur  1
CLex: A Lexicon for Exploring Color, Concept and
                                                                  Emotion Associations in Language
                                                                                  Svitlana Volkova, William Dolan, Theresa Wilson
I. Motivation                                                                                                                                                                                                                         III. Data Analysis
 People use color terms to describe visual characteristics of objects.                  Lexicon Statistics                                                                                                                   Top 10 Color Term Collocations for Berlin and Kay (1988) Colors

                                    sky - blue                         sun - yellow     CLex contains 15,200 annotations:                                                                                                      White – 0.62           Black – 0.43            Red – 0.59           Green – 0.54          Yellow – 0.63           Blue – 0.55
                                                                                          §  2,3K color term types                                                                                                           •  off, antique,      •  light,              •  dark, light,        •  dark, light,       •  light, dark,        •  light, sky,
                                    snow - white                       grass - green      §  1,9K concept types                                                                                                                 half, dark,           blackish               dish brown,            olive,                green, pale,           dark, royal,
                                                                                                                                                                                                                                 black, bone,          brown,                 brick,                 yellow, lime,         golden,                navy, baby,
                                                                                          §  3,4K affect term/emotion types                                                                                                     milky, pale,          brownish,              orange,                forest, sea,          brown,                 grey,
People take advantage of color terms to strengthen the message and                                                                                                                                                               pure, silver          brown, jet,            brown,                 dark olive,           mustard,               purple,
                                                                                                                                                                                                                                                       dark, green,           indian, dish,          pea, dirty            orange,                cornflower,
convey emotion in natural interaction.                                                  Color dimensions:                                                                                                                                              off, ash,              crimson,                                     deep, bright           violet
                                                                                          §  hue, darkness, brightness, saturation                                                                                                                    black grey             bright
Colors are indicative and have an effect on our feelings and emotions.                                                                NA                            Other                                                    *overall agreement on color terms (exact match – 0.49, substring – 0.46), free-marginal kappa (exact match – 0.46, substring – 0.42)
  §  positive emotions – joy, trust, admiration (Ortony et. al, 1988);                                                               8%                            12%

  §  negative emotions – aggressiveness, fear, boredom.                                                                                             Male
                                                                                                                                                                                    India
                                                                                                                                      Female
                                                                                                                                       51%
                                                                                                                                                     41%
                                                                                                                                                                            US
                                                                                                                                                                           38%
                                                                                                                                                                                    49%                                       Cross-Cultural Differences in Color-Emotion Associations: US and India, %
Color-concept-emotion associations [brown-darkness-boredom] vary
by culture and are influenced by the beliefs of the society:
  §  green – danger in Malaysia, envy in Belgium, happiness in Japan.                  Affect Terms
      (Sable and Akcay, 2010).                                                            §  feelings, emotions, attitudes, moods
                                                                                                                                                                                                                                                                                                         Trust
Some expressions involving colors share meaning across languages:




                                                                                                                                                                                                                anticipat.
                                                                                                                                                                                                                                                              Joy




                                                                                                                                                                                                surprise
                                                                                                                                                                                                                                                                                                         US: 33                                     Surprise




                                                                                                                                           sadness

                                                                                                                                                        disgust
  §  “read heat”, “blue blood”, “white collar”.                                                                                                                                                                                                           US: 331




                                                                                                                              anger
                                                                                                                                                                                                                                                                                                          I: 47	

                                   US: 18




                                                                                                                                                                                      trust
                                                                                                                                                                    fear
                                                                                                                                                                                                                                                            I: 154	





                                                                                                                                                                              joy
                                                                                                                                                                                                                                                                                                                                                      I: 12	

Deeper understanding of the associations between colors, concepts                                                   n    -            0.3             -          0.8       1.0      -         -            -
and emotions [red – blood – anger] may be helpful for:                                                              n   3.6          24.0           2.3          43.0      0.2      -         -            -
  §  Sentiment analysis: understanding emotion new article evokes.                                                 n   43.4          0.3           1.1          8.9       0.2     1.2        -            -
  §  Machine translation: as a source of paraphrasing.                                                             n   0.3           0.6           11.2         2.0       3.4     3.5       3.3          5.3
  §  Textual entailment: question answering in dialog systems                                                      n   0.3           0.3           1.1          1.2       5.7     1.2       6.7          5.3
  §  Human-computer interactions in real- and virtual word domains:                                                n   0.3           4.2           1.1          0.4       4.2     17.4      6.7           -
      online shopping, avatar construction in gaming environments                                                   n   3.3          11.4 24.7                   6.1       4.2     8.1       3.3          5.3
                                                                                                                                                                                                                                                       Anger                                             Sadness                                     Fear
      (clearer & natural descriptions “sky-blue shirt” vs. “blue shirt”).                                           n   0.6           0.3           1.1          0.4      9.1      1.2       3.3          5.3                                         US: 133                                           US: 171                                    US: 95
                                                                                                                    n   0.3           2.2           3.4          0.8       4.4     1.2       6.7           -                                           I: 160	

                                         I: 142	

                                 I: 105	

                                                                                                                    n   1.5           0.3           1.1          0.4       4.0     5.8       13.3 15.8
II. Data Collection                                                                                                 n   2.1          10.3             -          2.0       0.6     1.2       3.3          5.3

We use Amazon Mechanical Turk to collect color-concept and color-
emotion associations:                                                                   Basic Emotions in Colors, %                                                                                                          Top 10 Color-Concept Associations for Ambiguous Colors
                                                                                                                    8
                                         Task 1: showed swatch of RGB value                                         6
        Lips, 27            Hair,        Task 2: showed swatch + facial feature                                     4
                                                                                         Affect Term Polarity, %




                          eyebrow
                             36                                                                                     2
                                                                                                                    0
                                                                                                                    -2
  Facial
 mask, 27                                                                                                           -4
                                Eye
                              shadow,                                                                               -6
                                 26                                                                                 -8
        Skin, 18
                                         Q1. How would you name this color?                                        -10
                   Eyes, 18
                                         Q2. What emotions does this color evoke?                                  -12
    152 RGB values of facial             Q3. What concepts do you associate with it?
      features of avatars

Example annotations:                                                                   IV. Summary                                                                                                                                                                                   V. Related Work
  Ø  [R=222, G=207, B=186]
      light golden yellow è purity, happiness è butter cookie, vanilla               §  Constructed large-scale color-concept-emotion lexicon by crowdsourcing                                                                                                                   §    EmoLex (Mohammad, 2011)
      gold è cheerful, happy è sun, corn                                             §  Studied cross-cultural differences in color-emotion associations (US and India)                                                                                                          §    WordNet-Affect (Strapparava and Valitutti, 2004)
      golden è sexy è beach, jewelry                                                 §  Identified frequent color-concept associations (may help to identify sentiment                                                                                                           §    General Inquirer (Stone et. al., 1966)
  Ø  [R=218, G=97, B=212]                                                                 expressed by a concept)                                                                                                                                                                  §    Affective Forms of English Words (Stone et. al., 1966)
      pinkish purple è peace, tranquility, stressless è Justin Bieber’s
                                                                                       Acknowledgements
      headphones, someday perfume
                                                                                       §  Everyone in the NLP group at Microsoft Research for helpful discussion and feedback especially Chris Brocket, Piali Choudhury, and Hassan Sajjad.
      pink è happiness è rose, bougainvillea	

                                      §  Natalia Rud from Tyumen State University, Center of Linguistic Education for helpful comments and suggestions.
                                                                                       §  Johns Hopkins University Human Language Technology Center of Excellence.

Contenu connexe

Plus de Svitlana volkova

Plus de Svitlana volkova (18)

Grace Hopper Celebration 2010
Grace Hopper Celebration 2010Grace Hopper Celebration 2010
Grace Hopper Celebration 2010
 
Multimodal Information Extraction: Disease, Date and Location Retrieval
Multimodal Information Extraction: Disease, Date and Location RetrievalMultimodal Information Extraction: Disease, Date and Location Retrieval
Multimodal Information Extraction: Disease, Date and Location Retrieval
 
Web Intelligence 2010
Web Intelligence 2010Web Intelligence 2010
Web Intelligence 2010
 
Master Thesis
Master ThesisMaster Thesis
Master Thesis
 
MS Thesis Short
MS Thesis ShortMS Thesis Short
MS Thesis Short
 
IEEE ISI'10
IEEE ISI'10IEEE ISI'10
IEEE ISI'10
 
MedEx'10
MedEx'10MedEx'10
MedEx'10
 
Multilingual Ner Using Wiki
Multilingual Ner Using WikiMultilingual Ner Using Wiki
Multilingual Ner Using Wiki
 
WiML Poster
WiML PosterWiML Poster
WiML Poster
 
Topics Modeling
Topics ModelingTopics Modeling
Topics Modeling
 
Project Proposal Topics Modeling (Ir)
Project Proposal    Topics Modeling (Ir)Project Proposal    Topics Modeling (Ir)
Project Proposal Topics Modeling (Ir)
 
Social Networks
Social NetworksSocial Networks
Social Networks
 
Methods Of Reliability Analysis
Methods Of Reliability AnalysisMethods Of Reliability Analysis
Methods Of Reliability Analysis
 
Ohio Project
Ohio ProjectOhio Project
Ohio Project
 
Ukraine Presentation
Ukraine PresentationUkraine Presentation
Ukraine Presentation
 
Ukraine Presentation at Kansas State University
Ukraine Presentation at Kansas State UniversityUkraine Presentation at Kansas State University
Ukraine Presentation at Kansas State University
 
Communicatons Fulbright
Communicatons FulbrightCommunicatons Fulbright
Communicatons Fulbright
 
Communications Ternopil
Communications TernopilCommunications Ternopil
Communications Ternopil
 

Dernier

Textual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSTextual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSMae Pangan
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvRicaMaeCastro1
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQuiz Club NITW
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxAnupam32727
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...DhatriParmar
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesVijayaLaxmi84
 
4.11.24 Mass Incarceration and the New Jim Crow.pptx
4.11.24 Mass Incarceration and the New Jim Crow.pptx4.11.24 Mass Incarceration and the New Jim Crow.pptx
4.11.24 Mass Incarceration and the New Jim Crow.pptxmary850239
 
Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Celine George
 
4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptxmary850239
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfJemuel Francisco
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxkarenfajardo43
 
Active Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfActive Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfPatidar M
 
ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfVanessa Camilleri
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4JOYLYNSAMANIEGO
 
4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptxmary850239
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management SystemChristalin Nelson
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmStan Meyer
 

Dernier (20)

Textual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHSTextual Evidence in Reading and Writing of SHS
Textual Evidence in Reading and Writing of SHS
 
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnvESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
ESP 4-EDITED.pdfmmcncncncmcmmnmnmncnmncmnnjvnnv
 
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITWQ-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
Q-Factor HISPOL Quiz-6th April 2024, Quiz Club NITW
 
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptxCLASSIFICATION OF ANTI - CANCER DRUGS.pptx
CLASSIFICATION OF ANTI - CANCER DRUGS.pptx
 
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
Blowin' in the Wind of Caste_ Bob Dylan's Song as a Catalyst for Social Justi...
 
Faculty Profile prashantha K EEE dept Sri Sairam college of Engineering
Faculty Profile prashantha K EEE dept Sri Sairam college of EngineeringFaculty Profile prashantha K EEE dept Sri Sairam college of Engineering
Faculty Profile prashantha K EEE dept Sri Sairam college of Engineering
 
Sulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their usesSulphonamides, mechanisms and their uses
Sulphonamides, mechanisms and their uses
 
Paradigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTAParadigm shift in nursing research by RS MEHTA
Paradigm shift in nursing research by RS MEHTA
 
4.11.24 Mass Incarceration and the New Jim Crow.pptx
4.11.24 Mass Incarceration and the New Jim Crow.pptx4.11.24 Mass Incarceration and the New Jim Crow.pptx
4.11.24 Mass Incarceration and the New Jim Crow.pptx
 
Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17Tree View Decoration Attribute in the Odoo 17
Tree View Decoration Attribute in the Odoo 17
 
4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx4.11.24 Poverty and Inequality in America.pptx
4.11.24 Poverty and Inequality in America.pptx
 
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdfGrade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
Grade 9 Quarter 4 Dll Grade 9 Quarter 4 DLL.pdf
 
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptxGrade Three -ELLNA-REVIEWER-ENGLISH.pptx
Grade Three -ELLNA-REVIEWER-ENGLISH.pptx
 
Active Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdfActive Learning Strategies (in short ALS).pdf
Active Learning Strategies (in short ALS).pdf
 
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptxINCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
INCLUSIVE EDUCATION PRACTICES FOR TEACHERS AND TRAINERS.pptx
 
ICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdfICS2208 Lecture6 Notes for SL spaces.pdf
ICS2208 Lecture6 Notes for SL spaces.pdf
 
Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4Daily Lesson Plan in Mathematics Quarter 4
Daily Lesson Plan in Mathematics Quarter 4
 
4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx4.9.24 School Desegregation in Boston.pptx
4.9.24 School Desegregation in Boston.pptx
 
Transaction Management in Database Management System
Transaction Management in Database Management SystemTransaction Management in Database Management System
Transaction Management in Database Management System
 
Oppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and FilmOppenheimer Film Discussion for Philosophy and Film
Oppenheimer Film Discussion for Philosophy and Film
 

CLex Lexicon Reveals Cultural Differences in Color-Emotion Associations

  • 1. CLex: A Lexicon for Exploring Color, Concept and Emotion Associations in Language Svitlana Volkova, William Dolan, Theresa Wilson I. Motivation III. Data Analysis People use color terms to describe visual characteristics of objects. Lexicon Statistics Top 10 Color Term Collocations for Berlin and Kay (1988) Colors sky - blue sun - yellow CLex contains 15,200 annotations: White – 0.62 Black – 0.43 Red – 0.59 Green – 0.54 Yellow – 0.63 Blue – 0.55 §  2,3K color term types •  off, antique, •  light, •  dark, light, •  dark, light, •  light, dark, •  light, sky, snow - white grass - green §  1,9K concept types half, dark, blackish dish brown, olive, green, pale, dark, royal, black, bone, brown, brick, yellow, lime, golden, navy, baby, §  3,4K affect term/emotion types milky, pale, brownish, orange, forest, sea, brown, grey, People take advantage of color terms to strengthen the message and pure, silver brown, jet, brown, dark olive, mustard, purple, dark, green, indian, dish, pea, dirty orange, cornflower, convey emotion in natural interaction. Color dimensions: off, ash, crimson, deep, bright violet §  hue, darkness, brightness, saturation black grey bright Colors are indicative and have an effect on our feelings and emotions. NA Other *overall agreement on color terms (exact match – 0.49, substring – 0.46), free-marginal kappa (exact match – 0.46, substring – 0.42) §  positive emotions – joy, trust, admiration (Ortony et. al, 1988); 8% 12% §  negative emotions – aggressiveness, fear, boredom. Male India Female 51% 41% US 38% 49% Cross-Cultural Differences in Color-Emotion Associations: US and India, % Color-concept-emotion associations [brown-darkness-boredom] vary by culture and are influenced by the beliefs of the society: §  green – danger in Malaysia, envy in Belgium, happiness in Japan. Affect Terms (Sable and Akcay, 2010). §  feelings, emotions, attitudes, moods Trust Some expressions involving colors share meaning across languages: anticipat. Joy surprise US: 33 Surprise sadness disgust §  “read heat”, “blue blood”, “white collar”. US: 331 anger I: 47 US: 18 trust fear I: 154 joy I: 12 Deeper understanding of the associations between colors, concepts n - 0.3 - 0.8 1.0 - - - and emotions [red – blood – anger] may be helpful for: n 3.6 24.0 2.3 43.0 0.2 - - - §  Sentiment analysis: understanding emotion new article evokes. n 43.4 0.3 1.1 8.9 0.2 1.2 - - §  Machine translation: as a source of paraphrasing. n 0.3 0.6 11.2 2.0 3.4 3.5 3.3 5.3 §  Textual entailment: question answering in dialog systems n 0.3 0.3 1.1 1.2 5.7 1.2 6.7 5.3 §  Human-computer interactions in real- and virtual word domains: n 0.3 4.2 1.1 0.4 4.2 17.4 6.7 - online shopping, avatar construction in gaming environments n 3.3 11.4 24.7 6.1 4.2 8.1 3.3 5.3 Anger Sadness Fear (clearer & natural descriptions “sky-blue shirt” vs. “blue shirt”). n 0.6 0.3 1.1 0.4 9.1 1.2 3.3 5.3 US: 133 US: 171 US: 95 n 0.3 2.2 3.4 0.8 4.4 1.2 6.7 - I: 160 I: 142 I: 105 n 1.5 0.3 1.1 0.4 4.0 5.8 13.3 15.8 II. Data Collection n 2.1 10.3 - 2.0 0.6 1.2 3.3 5.3 We use Amazon Mechanical Turk to collect color-concept and color- emotion associations: Basic Emotions in Colors, % Top 10 Color-Concept Associations for Ambiguous Colors 8 Task 1: showed swatch of RGB value 6 Lips, 27 Hair, Task 2: showed swatch + facial feature 4 Affect Term Polarity, % eyebrow 36 2 0 -2 Facial mask, 27 -4 Eye shadow, -6 26 -8 Skin, 18 Q1. How would you name this color? -10 Eyes, 18 Q2. What emotions does this color evoke? -12 152 RGB values of facial Q3. What concepts do you associate with it? features of avatars Example annotations: IV. Summary V. Related Work Ø  [R=222, G=207, B=186] light golden yellow è purity, happiness è butter cookie, vanilla §  Constructed large-scale color-concept-emotion lexicon by crowdsourcing §  EmoLex (Mohammad, 2011) gold è cheerful, happy è sun, corn §  Studied cross-cultural differences in color-emotion associations (US and India) §  WordNet-Affect (Strapparava and Valitutti, 2004) golden è sexy è beach, jewelry §  Identified frequent color-concept associations (may help to identify sentiment §  General Inquirer (Stone et. al., 1966) Ø  [R=218, G=97, B=212] expressed by a concept) §  Affective Forms of English Words (Stone et. al., 1966) pinkish purple è peace, tranquility, stressless è Justin Bieber’s Acknowledgements headphones, someday perfume §  Everyone in the NLP group at Microsoft Research for helpful discussion and feedback especially Chris Brocket, Piali Choudhury, and Hassan Sajjad. pink è happiness è rose, bougainvillea §  Natalia Rud from Tyumen State University, Center of Linguistic Education for helpful comments and suggestions. §  Johns Hopkins University Human Language Technology Center of Excellence.