Critique is an important component of creative work in design education and practice, through which individuals can solicit advice and obtain feedback on their work. Face-to-face critique in offline settings such as design studios has been well-documented and theorized. However, little is known about unstructured distributed critique in online creative communities where people share and critique each other’s work, and how these practices might resemble or differ from studio critique. In this paper, we use mixed-methods to examine distributed critique practices in a UX-focused online creative community on Reddit. We found that distributed critique resembles studio critique categorically, but differs qualitatively. While studio critique often focuses on depth, distributed critique often revolved around collective sensemaking, through which creative workers engaged in iteratively interpreting, defining, and refining the artifact and their process. We discuss the relationship between distributed critique and socio-technical systems and identify implications for future research.

1. YUBO KOU
Florida State University
SUPPORTING DISTRIBUTED CRITIQUE
THROUGH INTERPRETATION AND
SENSE-MAKING IN AN
ONLINE CREATIVE COMMUNITY
COLIN M. GRAY
Purdue University

2.
CRITIQUE?
WHAT DOES IT MEANTO

3.
CRITIQUE?
WHAT DOES IT MEANTO
STUDIO PROFESSIONAL HYBRID CROWD

4. “a dialogue in which the interjection
of a reasoned opinion about a product
or action triggers further
reflection on or changes to the
artifact being designed”
Gerhard Fischer, Kumiyo Nakakoji, Jonathan Ostwald, Gerry Stahl, andTamara Sumner. 1993.
Embedding critics in design environments. Knowl. Eng. Rev. 8, 4, 285–307.

5. PHYSICAL

6. PHYSICAL DISTRIBUTED

7. DESIGN
PRESENTER
Why registration rate went down after we tested social
login buttons? […] We tested out “create account with
FB” button on order confirmation page and registration
dropped by 50%, compared to the past. Any idea why?

8. DESIGN
PRESENTER
Why registration rate went down after we tested social
login buttons? […] We tested out “create account with
FB” button on order confirmation page and registration
dropped by 50%, compared to the past. Any idea why?
CRITIC
The user was ok as long as there was no
commitment. They would be annoyed if you asked for
a commitment by using their FB account.

9. DESIGN
PRESENTER
Why registration rate went down after we tested social
login buttons? […] We tested out “create account with
FB” button on order confirmation page and registration
dropped by 50%, compared to the past. Any idea why?
CRITIC
The user was ok as long as there was no
commitment. They would be annoyed if you asked for
a commitment by using their FB account.
DESIGN
PRESENTER
Agree. The timing is bad to ask users to connect with a
social media account when they want to leave!

10. DESIGN
PRESENTER
Why registration rate went down after we tested social
login buttons? […] We tested out “create account with
FB” button on order confirmation page and registration
dropped by 50%, compared to the past. Any idea why?
CRITIC
The user was ok as long as there was no
commitment. They would be annoyed if you asked for
a commitment by using their FB account.
DESIGN
PRESENTER
Agree. The timing is bad to ask users to connect with a
social media account when they want to leave!
CRITIC You got it!

11. DESIGN
PRESENTER
Why registration rate went down after we tested social
login buttons? […] We tested out “create account with
FB” button on order confirmation page and registration
dropped by 50%, compared to the past. Any idea why?
CRITIC
The user was ok as long as there was no
commitment. They would be annoyed if you asked for
a commitment by using their FB account.
DESIGN
PRESENTER
Agree. The timing is bad to ask users to connect with a
social media account when they want to leave!
CRITIC You got it!
DESIGN
PRESENTER
Do you happen to know some books or
theories supporting this idea?

12. Judgment: an assessment of quality from the perspective of the critic
Process-Oriented: statement or question regarding the design approach or
processes that led to a design artifact
Brainstorming: statement or question about imagined possibilities or next
steps, such as “what-if” questions
Interpretation: critic telling what they see and how they make sense of the
design artifact
Direct Recommendation: providing specific, targeted advice to improve a
specific aspect of design
Investigation: non-rhetorical questions about the design or its processes
Free Association: critic’s initial reactions to a design, such as “it looks like…”
statements
Comparison: comparison or contrast with an external artifact, concept, or idea
Identity Invoking: reference to student’ identity as a designer in relation to a
future professional community
Deanna P. Dannels and Kelly Norris Martin. 2008. Critiquing critiques: a genre analysis of feedback across
novice to expert design studios. J. Bus.Tech. Commun. 22, 2, 135–159.
TYPOLOGY

13. /r/userexperience
January–August 2016
970
POSTS
6,958
COMMENTS

14. 970
POSTS
94
POSTS
Critique-
oriented
6,958
COMMENTS
864
COMMENTS
90
POSTS
Excluding
special threads
569
COMMENTS
Original
corpus

15. 1) What design artifacts do community
members offer for critique?
2) How does distributed critique
content differ from studio critique?
3) How do discourse practices enable
critique and advice-giving to occur
in this online community?

16. 1. What design artifacts do
community members offer for critique?

17. PREPARATION
Interest in UX
characteristics of their
project, but lack
relevant expertise or
skills to move forward.
11
IDEATION
Seeking inspiration for a
project that was already
underway.
20
WORK-IN-PROGRESS
Encountering an issue
with a specific feature
or element in an in-
progress artifact.
46
FINAL WORK
Presenting final artifacts
to be critiqued.
13

18. 2. How does distributed critique
content differ from studio critique?

19. PHYSICAL (D&M) % REDDIT %
Judgment 25% 16%
Process-Oriented 21% 26%
Brainstorming 18% 14%
Interpretation 12% 41%
Direct Recommendation 9% 21%
Investigation 5% 17%
Free Association 4% 4%
Comparison 3% 19%
Identity Invoking 3% 5%
Phatic - 4%

20. PHYSICAL (D&M) % REDDIT %
Interpretation 12% 41%
• User-related: Customers can be generally happy with
inconsistency across parts of a product.
• Design-related: It may be selected by an algorithm but it
makes the selection seem more meaningful
• Creator-related: The reason why you can’t tell which color
to use is because you haven’t figured out the correct task
to solve.

21. PHYSICAL (D&M) % REDDIT %
Comparison 3% 19%
• Creative knowledge: I think you might need a [skip link]
(http://webaim.org/techniques/skipnav).
• Similar product: MailChimp does an even better job of
this on mobile (https://login.mailchimp.com/signup).
• Creator: A friend of mine worked on a play with a human
actor and a robot arm.

22. 0%
12.5%
25%
37.5%
50%
Judgment Process-Oriented Brainstorming Interpretation Direct Rec. Investigation Free Association Comparison Identity Invoking Phatic Comm.
Preparation Ideation W.I.P Final

23. NumberofCritiques
0
125
250
375
500
Times of Replies
1 2 3 4 5 6 7 8 9
Judgment Process-Oriented Brainstorming Interpretation Direct Recommendation
Investigation Free Association Comparison Identity Invoking Phatic Communication

24. 3. How do discourse practices enable
critique and advice-giving to occur
in this online community?

25. INVESTIGATION INTERPRETATION INVESTIGATION

26. Post title: How to accommodate e-commerce for blind users? Specifically the checkout button...
Post content: I’m facing a design challenge right now – the checkout button on a site is a shopping
cart icon located in the header. A user that is using a screen reader would have to navigate all the
way back to it – in order to checkout. Aside from putting checkout buttons everywhere (which
cannibalize the shopping cart icon in the header, and also just comes across as tacky/pushy) how can
we ensure blind users have a quick path to checkout? Any ideas or resources would be appreciated!
Critic: What do you mean the blind user will have to navigate all the way back to it to check out?
Poster: Since the checkout icon is in the header – and the item they’ve added is in the body, theyd
have to navigate (using an audio interface) back to the header, in order to locate the checkout button
- which is actually inside an overlay/fly out/drawer - that opens when you select the shopping cart
icon (think amazon).
Critic: Do users check out from a list view (i.e. Add a bunch of items at once, and check out) Or from
a product page (I.e. Add one item at a time)? I imagine it's the latter. Why can't you have a
"proceed to checkout" button on the product pages?
INVESTIGATION INTERPRETATION INVESTIGATION

27. INTERPRETATION DIRECT RECOMM. PROCESS-ORIENTED+ +
INTERPRETATION

28. Post title: Get users to read | signup form @ physical location
Post content: Hey! We run a VR lounge in Vienna. So basically a bar where you can use virtual reality
equipment…. This is displayed on two giant tablets at a part of the bar… when the signup button is
pressed, this screen appears…
Critic: A couple of things:
* As /u/vcarl said, people don't read. So you'll need to...
* Put things in steps. Make them each exactly one screen tall and one screen wide. […]
* If at all possible, don't make them log in after they sign up. Many use cases will require a log-in
action, but if at all possible, try to do away with the create account -> validate account -> log
into the account I just created and validated loop. […]
Poster: thank you for your reply. * My first version was based on steps, which resulted in four
individual screens with giant "NEXT" buttons. But somehow people didnt know what to do […] * I
dont think I understand what you mean. After "create account" the user is done. What do you mean
with validate and log in? Jeah, come visit if you can :)
INTERPRETATION DIRECT RECOMM. PROCESS-ORIENTED+ +
INTERPRETATION

29. DISCUSSION & IMPLICATIONS

30. SOCIAL
LEARNING
CRITIQUE AS
DIALOGUE
GLOBAL
INTERACTION

31. SOCIAL
LEARNING
• Promoting sharing of UX Knowledge
• Related to disciplinary knowledge production
• Aids in concretizing UX as a discipline

32. CRITIQUE AS
DIALOGUE
• Foregrounds social dimensions of critique
• Highlights contextually and collective sense making
• Design activity as dynamic and evolving

33. GLOBAL
INTERACTION
• Sociality to support and sustain creative labor
• Engagement with identity construction and learning
• Reliance on phatic communication

34. YUBO KOU
yubokou.info
ykou@fsu.edu
COLIN M. GRAY
colingray.me | uxp2.com
gray42@purdue.edu
THANK YOU