This is our experiment in crowd-sourcing knowledge acquisition for universal truth creation out of complicated textbook sentences. An integration of Project Halo's SMW and AURA efforts.

1. Subject Matter Expert - User Experience Review
AuraWiki
Jesse Wang, Wil Smith
Oct 2012

2. Project Goals
“Crowd Source UT Authoring”
Can Subject Matter Experts Author Useful
Universal Truths?
Can We Speed Up Encoding a Textbook with Input
from Subject Matter Experts?
Can We Create a UT Authoring Portal for Multiple
Textbooks?
Can Existing Social Networks Provide Subject
Matter Experts Capable of UT Authoring?
Could Gamification be Applied to An Existing
Portal to Add Non-Domain Experts?

3. Meeting Goals
This is an end user meeting and we are doing it to hear
your comments on the current system.
The team wants to explain the authoring process, and
demonstrate the current application state.
Before we continue development on the project we need
to review how the application, in the unfinished state,
performs with our targeted user base.
MID PROJECT FEEDBACK. We do these meetings when
releasing production products to ensure the final
application complies with customer needs and is not “the
engineering best way”.

4. Review Schedule
Phase Method Time required
Introduction & Training • Goals of Project Halo and Aura 0:45
Wiki
• Training for UT Authoring
• What is a UT?
• What is a concept / context?
• How do I formulate UTs
• System demonstration & hands-on
Assignment: UT Authoring • You work by yourself using our 1:30
system
• Support is provided by us
Questionnaire & Discussion • Online questionnaire 0:45
• Group discussion

5. Subject Matter Expert - User Experience Review
How do we author a
universal truth?

6. From Sentence to
Universal Truths
What is a UT?
How to write UTs for a sentence
What it a concept?
How to choose concepts for a UT

7. What is a UT?
There is no exact definition of “Universal Truth”, but a good UT…
expresses a single fact (which can be compound, but must be indivisible).
is as simple as possible, but not simpler (to quote Einstein).
is context-independent, i.e. meaningful on its own, without surrounding
sentences or other context.
is universally true. For example, a UT could start with "All Plants..." but not
with "Some plants...".
is unambiguous.
is precise.

8. How to Write UTs for a
Sentence
For each relevant sentence, write one or more UTs
There is no one correct set of UTs for a given sentence.
The source sentences are open to interpretation, and it is
up to you to determine what the most appropriate UTs are
The interpretation of the source sentences, and the
paraphrasing into UTs, should be guided by your
knowledge of the subject matter (and by common sense)
The following methods are not exhaustive. When in
doubt, refer back to the definition of a “good UT”.

9. How to Write UTs for a
Sentence
In the simplest case, the sentence has only one UT, and it
is exactly the same as the UT.
Sentence: Matter is made up of elements.
UT: Matter is made up of elements.

10. How to Write UTs for a
Sentence
Sometimes part of a sentence is irrelevant, even though
the sentence as a whole has been deemed relevant. In
such cases the irrelevant part is simply dropped.
Sentence: Some symbols are derived from Latin or
German; for instance, the symbol for sodium is Na.
UT: Sodium has the symbol Na.

11. How to Write UTs for a
Sentence
Often, a sentence contains several facts. In this case, the
sentence needs to be split into several UTs.
Sentence: Pure sodium is a metal, and pure chlorine is a
poisonous gas.
UT: Pure sodium is a metal.
UT: Pure chlorine is a poisonous gas.

12. How to Write UTs for a
Sentence
When a reference like "this", "that", "these", "those", "which", etc
occurs in a source sentence, they should be replaced with the
referent (the word that "this","that", etc refers to).
Sentence: Organisms are composed of matter, which is defined as
anything that takes up space and has mass
UT: Organisms are composed of matter
UT: Matter has mass and volume
Here, "which" clearly refers to "Organisms", and "is defined as" is
unnecessarily verbose, and thus simplified.

13. How to Write UTs for a
Sentence
Sometimes the referent is in a previous sentence in the
same paragraph.
Sentence: These selective channels affect the rate at
which water moves osmotically across the membrane
UT: Aquaporins are selective channels
UT: Aquaporins affect the rate at which water moves
osmotically across the membrane

14. How to Write UTs for a
Sentence
There are many other words that are used as linguistic flourishes or to bind sentences in a
paragraph together and give the text some "flow". Such words ("another" in the example
below) should be eliminated, but can sometimes be used to extract or infer "incidental"
information from a sentence.
Sentence: Water (H20), another compound, consists of the elements hydrogen (H) and
oxygen (O) in a 2:1 ratio
UT: Water consists of hydrogen and oxygen in a 2:1 ratio
UT: Water is a compound
UT: Hydrogen has symbol H
UT: Oxygen has symbol O
Here, the first UT contains the "main" information of the sentence, and the last three are
"incidental".

15. How to Write UTs for a
Sentence
Ideally, a UT should not contain words like "sometimes",
"usually", "most", "some", "generally", etc. There are a
couple of ways to handle them. One way is to generalize the
weaker statement into a universal statement, usually just by
dropping the word in question.
Sentence: Branching generally enables plants to harvest
sunlight for photosynthesis more effectively.
UT: Branching enables plants to harvest sunlight for
photosynthesis more effectively.

16. How to Write UTs for a
Sentence
Another method of eliminating "some" etc is to instantiate the
sentence by providing a name for the case where the sentence is
true.
Sentence: Some prokaryotes even contain a row of tiny magnetic
particles that allow the cells to orient in Earth's magnetic field
UT: Magnetotactic bacteria contain a row of tiny magnetic particles
that allow the cells to orient in Earth's magnetic field
Here we replaced "some prokaryotes" with "magnetotactic
bacteria". We only do this when the new name occurs somewhere in
the textbook.

17. How to Write UTs for a
Sentence
"some", etc are allowed when they do not refer to the main concept of
the sentence (see "Choosing Concepts for a UT", below). For example,
this is a good UT:
UT: In vascular plants, some substances may use more than one route
The UT as a whole states a fact about all vascular plants (the main
concept), and "some" refers to substances within all vascular plants.
One way to remember this distinction is to keep in mind that Universal
Truths are just that -- universally true statements, i.e. statements that
are always true of some type of entity, event, etc. If it is possible, we
can also use Instantiation (see previous slide) in this type of case.

18. How to Write UTs for a
Sentence
When "for example", "such as", etc occurs in a sentence,
encode the specific information in the example.
Sentence: Some plants have horizontally oriented leaves;
others, such as grasses, have leaves that are vertically
oriented.
UT: Grass has vertically oriented leaves
As discussed above, the "some" part of the sentence is
problematic.

19. How to Write UTs for a
Sentence
In other cases, the "for example" does not add any new
information.
Paragraph: We symbolize atoms with the same abbreviation
used for the element that is made up of those atoms. For
example, the symbol C stands for both the element carbon
and a single carbon atom.
Here, the second sentence does not add any information,
since its informational content is subsumed by the general
"rule" of the first sentence (assuming we have already
encoded that carbon has the symbol C).

20. How to Write UTs for a
Sentence
Conditional statements should usually be rephrased to a non-
conditional form.
Sentence: If we place a cell in a solution that is hypotonic to the cell,
water will enter the cell faster than it leaves
UT: A cell in a hypotonic solution will gain water faster than it loses
it
Here, the condition has been changed into a context, "in a
hypotonic solution". See also the section on "Contextual
Universals" below.

21. Review: What is a UT?
Or rather a good UT…
expresses a single fact (which can be compound, but must be indivisible).
is as simple as possible
is context-independent, i.e. meaningful on its own, without surrounding
sentences or other context.
is universally true. For example, a UT could start with "All Plants..." but not
with "Some plants...".
is unambiguous.
is precise.

22. Summary: Methods of
UT Authoring
Copy, remove (partial, irrelevant info), splitting
Reference resolution (from context)
Incidental extraction
e.g. “Water, another compound, …”
Specialization, generalization
Instantiation
E.g. “some prokaryotes”  “magnetotactic bacteria”
Example-encoding
E.g. “some plants…”  “grass”
Condition to context

23. What is a Concept?
A concept is a noun or a noun phrase that describes a
type of entity or event, in singular form
You may need to create a new concept for a UT if an
appropriate one does not already exist.
Good concept names: Cell, Eukaryotic Cell, Cellular
Respiration
Bad concept names: Cells, The eukaryotic cell, A plant

24. Choosing Concepts for a
UT
The main concept of a UT is the entity or event that the UT is about.
It is usually the grammatical subject of the UT
UT: All plant cells have a nucleus
Main concept: Plant Cell
Note that "All plant cells" is not a good concept, because it is not in singular
form.
UT: Photosynthesis produces oxygen
Main concept: Photosynthesis

25. Choosing Concepts for a
UT
Some UTs are true in a specific context. In such cases, also choose a contextual concept –contextual concept is OPTIONAL
The contextual concept usually follows a word such as "in", "during", "while", etc.
UT: The phospholipids in a membrane move laterally
Main concept: Phospholipid
Contextual concept: Membrane
Note that "the" or "a" is never part of the concept.
UT: The oxygen produced during photosynthesis is used during cellular respiration
Main concept: oxygen
Contextual concept: Photosynthesis
UT: Cellular respiration in eukaryotes happens in the mitochondria
Main Concept: Cellular Respiration
Contextual Concept: Eukaryote

26. Bring your Best, then
Move On
We hope you got good understanding of what you’re
supposed to do
Give it your best shot, it does NOT need to be perfect
Do not spend too much time on one sentence/UT,
move on if it’s too hard
Feel free to work on sentences or UT others started
Feel free to ask us if you’re really confused or not
sure

27. Subject Matter Expert - User Experience Review
Aura wiki tutorial

28. Our Solution: Aura Wiki
Contents:
A few chapters of Campbell Biology
Without figures and using a simplified layout
Purpose
Use the power of crowdsourcing and online
collaboration
Let users „translate“ sentences from the textbook into
UTs.

29. Sample wiki page for the
textbook

30. Sample wiki page for a whole
section

31. Sample wiki page for a
sentence

32. Creating UTs
The translation of sentences to UTs is done on the
sentence page.
The sentence page can display the paragraph
surrounding the sentence (or even more context) if
needed for understanding and formulating the UT.
Under the heading Universal Truths you can see all
UTs that exist for this sentence and you can add yor
own.

33. Sample UTs presented in the sentence page

34. Creating UTs – Add new
UT

35. Creating UTs – Input
Form
Input form for new UT.
First two inputs are
required.

36. Creating UTs -
Autocompletion
Autocompletion is
automatically triggered
and suggests existing
concepts

37. Subject Matter Expert - User Experience Review
Subject Matter Expert
Assignment

38. The Assignment
Now it is your turn
Go to your User page
There is a list of sentences assigned to you
Once you have added UTs they will be displayed there
together with the sentences (after refreshing the page)
At the top of your user page, you will find links to help-
texts, some sample UTs and the questionnaire for after
the session.

39. The Assignment
This link always
brings you back to
your user page
Help documents
and a list of
example UTs
Sentences
(and UTs)

40. The Assignment
IMPORTANT
This evaluation is conducted in order to assess the system, not to
assess your work as a SME. You are supporting the evaluation, but
you are not a test subject!
Please work with the system at your own convenience. You do not
need to achieve any quantitative or qualitative goal. (This is of
course no excuse for being lazy )
You have pencils and paper available – if you encounter any
problems or have ideas you think are worth mentioning, feel free
to take a short note so you can bring it up during the discussion
later
If there are questions please ask them anytime

41. Subject Matter Expert
Subject Matter Expert - User Experience Review
Review

42. Questionnaire
Go to your user page
There is a link to the online questionnaire at the top
In the first question, select the username you were
assigned today
Fill out the entire questionnaire. Don’t forget to
submit at the end

43. Discussion – System
Functionality
Did you have trouble following the UT authoring
workflow? If so, where and why?
What changes would you apply to the existing
functionality and/or workflows to increase productivity
and ease of use?
What new functionality could you think of that might have
a positive influence on your work?

44. Discussion – UT
Authoring
Did you understand the notion of UT and concept?
Did the training sufficiently explain the meaning of UT and
concept?
Do you feel the training you received in the beginning and
the help materials on the site were sufficient for
executing your task? If not, what would you improve?
Which areas need better coverage?

45. Subject Matter Expert - User Experience Review
Conclusion

46. Thank you.
Contact: Jesse Wang (jessew@vulcan.com)
William Smith (wills@vulcan.com )