1. Processing Natural Language in Robotics Applications 01001000 01100101 01101100 01101100 01101111 00100000 01010111 01101111 01110010 01101100 01100100 00100001 00100000 Alan Shen

2. Related work Wubble Voice Command Demo (Gazebo Simulation) Arizona Robotics Research Group - University of Arizona http://www.youtube.com/watch?v=atB9mh6u1Ng http://ua-ros-pkg.googlecode.com

3. Related work Humanoid robot speech recognition and object tracking http://www.youtube.com/watch?v=0jW9LgtiiM8

4. Goals Implement sentence recognition in text form (console/gui) Provided some basis for speech processing “Pick up the blue cup” Pick up the blue cup

5. Existing tools Possible python library implementation Open Rave Text Processing Speech Processing Robot Action Prairie Dog Libraries Python Tagging Libraries (eg: NLTK)

6. Parsing orders Ambiguous interpretations (robot command sentences may suffer less from this) “Robot, make her duck” “Get the elevator” Mapping verbs to targets: “Follow that person” “Get in the elevator” “Pick up that object”

7. Parts of speech tagging Given a word, what is its part of speech? 𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠)

8. Parts of speech tagging 𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠) Given a known corpus, maybe it’s easier to predict a word given a tag: Bayes: 𝑃𝐴𝐵=𝑃𝐵𝐴𝑃(𝐴)𝑃(𝐵)

9. Parts of speech tagging 𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠) Bayes: 𝑃𝐴𝐵=𝑃𝐵𝐴𝑃(𝐴)𝑃(𝐵) 𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃𝑊𝑜𝑟𝑑𝑠𝑇𝑎𝑔𝑠𝑃(𝑇𝑎𝑔𝑠)𝑃(𝑊𝑜𝑟𝑑𝑠) Don’t need to normalize…

10. Parts of speech tagging 𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠) Bayes: 𝑃𝐴𝐵=𝑃𝐵𝐴𝑃(𝐴)𝑃(𝐵) 𝐴𝑟𝑔𝑀𝑎𝑥:𝑃𝑊𝑜𝑟𝑑𝑠𝑇𝑎𝑔𝑠𝑃𝑇𝑎𝑔𝑠

11. Calculating ArgMax and likely tags Speech and Language Processing - Jurafsky and Martin

12. Further work Once actions and their targets are mapped, generate actions Directly executing OpenRave commands in sequence? Commands ignored if OpenRave is busy Determine method of controllingPrairieDog movement Need to ensure that text command interfaceis compatible with both arm and wheel controls Populate objects in robot’s word dictionary Eg: block, door, elevator, person

13. Questions? Thank you!