The document is a paper from the Color Imaging XVII conference that discusses algorithms for determining harmonious color palettes from images on the internet. It proposes harvesting images to categorize colors, annotating them with metadata, storing the data in a database, and developing a cloud-based application to query the database and return harmonious color palette suggestions based on keywords or samples. Current work involves using Wikimedia images, developing color categorization techniques, and prototyping mobile apps to test the system.

1. Harmonious colors: from alchemy to science
Giordano B. Beretta Nathan M. Moroney
Printing & Content Delivery Lab
Hewlett-Packard Laboratories
Palo Alto, California
Color Imaging XVII:
Displaying, Processing, Hardcopy, and Applications
San Francisco Airport, 24 – 26 January 2012
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 1 / 12

2. Color palettes cannot be assembled color by color
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 2 / 12

3. Historical palettes can be reconstructed
. . . but today there are no limits on color specification other than gamut hulls
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 3 / 12

4. Color theories are tempting
1070-Y Gelb
1070-Y30R Gelborange
2070-G50Y Gelbgrün
3060G 1070-Y50R Grün Orange
Blau- Rot-
4040-B50G 1070-Y80R grün orange
4040B 2060R Blau Rot
Rotviolett
3560-R80B Blauviolett
2060-R20B
3055-R50B Violett
. . . but symmetries in a color space are not useful in practice, other than for
teaching
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 4 / 12

5. Color palettes by professional designers
more recent: Adobe Kuler, COLOURLovers, MTurk
designer’s palettes are typically small, how can I extend a palette?
how can I find the best palette in a large collection?
how do I know a palette is still en vogue?
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 5 / 12

6. The many meanings of harmony
no visual mechanism for color harmony
prophet / philosopher harmony = go well together, an informal
concept
alchemist: no scientifically rooted basis
archetypes for a tool
archetype: palette engrained in a
culture’s collective mind, based on
harmony / colorant availability, etc.
color consultant: capture the sentiment
Far East: harmony = people = sum
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 6 / 12

7. The third Millennium
StorageWorks MSA1000
UPS 3000 XR
DL360
cloud computing data mining
crowd-sourcing sentiment analysis
big data everything has to scale
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 7 / 12

8. From West to Far East
new concept of harmonious color palette as a collective sentiment
crawl the internet for data and annotate it with metadata
reduce the information through categorization
store and maintain knowledge on color palettes in the cloud
suggest small numbers of palettes though a simple application
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 8 / 12

9. Algorithm
1. harvest images from the Internet
cull icons, logos, people, . . .
2. categorization
quantize images lexically by color terms
weigh colors by area, saliency or regions of interest
3. annotate
collect simple metadata like date and geolocation
harvest keywords by parsing captions and surrounding text
4. store in a database
periodically update by repeating steps 1–3
add usage data from clients
provide a feedback mechanism to the end-users
5. simple cloud application
query service by keywords or sample image
query result is clustered (greyworld!) and top 7 ± 2 palettes are served
collect feedback from end-users
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 9 / 12

10. Current status
use Wikimedia images
color categorization using Moroney and xkcd corpora
analyzing robustness
studying scalability and asymptotic behavior
prototype: mobile application on TouchPad, Android, iPhone
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 10 / 12

11. Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 11 / 12

12. Definition of sentiment
a feeling can be almost any subjective reaction or state—pleasant or
unpleasant, strong or mild, positive or negative—that is characterized by
an emotional response (a feeling of insecurity; a feeling of pleasure).
an emotion is a very intense feeling, which often involves a physical as
well as a mental response and implies outward expression or agitation (to
be overcome with emotion).
passion suggests a powerful or overwhelming emotion, with
connotations of sexual love (their passion remained undiminished after
30 years of marriage) or intense anger (a passion for revenge).
there is more intellect and less feeling in sentiment, which is often
applied to an emotion inspired by an idea (political sentiments; antiwar
sentiments). Sentiment also suggests a refined or slightly artificial
feeling (a speech marked by sentiment rather than passion).
affect is a formal psychological term that refers to an observed emotional
state (heavily sedated, he spoke without affect).
Beretta & Moroney (HP Labs) The Dark Side of the Color EI 2012 paper 8292-17 12 / 12