SlideShare une entreprise Scribd logo
1  sur  6
RoboCart: Toward Robot-Assisted Navigation of Grocery
                Stores by the Visually Impaired∗
                                   Vladimir Kulyukin            Chaitanya Gharpure        John Nicholson
                                            Computer Science Assistive Technology Laboratory
                                                   Department of Computer Science
                                                         Utah State University
                                                          Logan, Utah, U.S.A
                                               {vkulyukin, cpg, jnicholson}@cc.usu.edu


   Abstract— This paper presents RoboCart, a proof-of-concept
prototype of a robotic shopping assistant for the visually
impaired. The purpose of RoboCart is to help visually impaired
customers navigate a typical grocery store and carry purchased
items. The hardware and software components of the system
are presented. For localization, RoboCart relies on RFID tags
deployed at various locations in the store. For navigation, Robo-
Cart relies on laser range finding. Experiences with deploying
RoboCart in a real grocery store are described. The current
status of the system and its limitations are outlined.

  Index Terms— service robotics

                        I. I NTRODUCTION
   Grocery shopping is a routine activity that people all over
the world perform on a regular basis. However, grocery stores
and supermarkets remain largely inaccessible to people with
visual impairments. The main barrier is the inadequacy of the
principal navigation aids, such as guide dogs and white canes,
for negotiating the seemingly simple topological structure
of a typical grocery store. While guide dogs are helpful in                                Fig. 1.   RoboCart in a grocery store.
micro-navigation, e.g., local obstacle avoidance and homing
in on simple targets such as exit doors, they offer little
assistance with macro-navigation, which requires topological                the grocery shopping for the visually impaired. Rather, the
knowledge of the environment. A guide dog may pick up                       purpose is to assist visually impaired customers in navigating
and remember routes to a small set of grocery items through                 the store and carrying the purchased items around the store
repeated exposure to those routes. However, the guide dog                   and to the check-out registers.
cannot help its handler in a situation where the store changes                This paper describes RoboCart, a proof-of-concept proto-
the location of several items between visits or stops carrying              type of a robotic grocery shopping assistant for the visually
an item. Nor can the dog assist its handler with pushing a                  impaired. RoboCart is shown in Fig. 1. The paper is orga-
shopping cart. Needless to say, the white cane does not fare                nized as follows. First, related work is reviewed. Second,
any better under the same circumstances.                                    RoboCart’s software and hardware components are presented.
   In August 2004, the Assistive Technology Laboratory of                   Third, the scope and feasibility issues are discussed. Finally,
the Department of Computer Science (CS) of Utah State                       the current status of the system is described through experi-
University (USU) and the USU Center for Persons with                        ences with deploying RoboCart in a real grocery store.
Disabilities (CPD) started a collaborative project whose ob-
jective is to build a robotic shopping assistant for the visually                                II. R ELATED W ORK
impaired. The purpose of the shopping assistant is not to do                   Several researchers have developed robotic guides for
  ∗ This work is partially supported by NSF Grant IIS-0346880 and two
                                                                            indoor environments. While these systems are not explicitly
Community University Research Initiative (CURI) grants (CURI 2004 and       designed for the visually impaired, they offer valuable fea-
CURI 2005) from the State of Utah. All grants are awarded to V. Kulyukin.   sibility studies in robotic navigation of indoor environments.
Horswill [6] built Polly, a small robotic guide for the MIT
AI Lab. Polly used lightweight vision routines that depended
on textures specific to the lab. Burgard et al. [2] developed
RHINO, an interactive tour guide that was deployed in
a museum in Germany for a period of six days. Thrun
et al. [16] used the approaches developed in RHINO to
build MINERVA, an autonomous tour-guide robot that was
deployed in the National Museum of American History in
Washington, D.C.
   Borenstein and Ulrich introduced GuideCane [7], a mobile
obstacle avoidance device for the visually impaired. Guide-
Cane consists of a long handle and a sensor unit mounted
on a steerable two-wheel axle. The sensor unit consists of
ultrasonic sensors that detect obstacles and help the user
to steer the device around them. A steering servo motor,
operating under the control of the onboard computer, steers                    Fig. 2.   RFID tag in a grocery store.
the wheels relative to the handle.
   The Haptica Corporation developed Guido, a robotic walk-
ing frame for people with impaired vision and reduced           signal from RF tags and uses triangulation to estimate their
mobility [17]. Guido uses the onboard sonars to scan the        positions. Another RFID-based navigation system for indoor
immediate environment for obstacles and communicates de-        environments was developed at the Atlanta VA Rehabilitation
tected obstacles to the user via speech synthesis. Mori and     Research and Development Center [13], [14]. In this system,
Kotani [12] developed HARINOBU-6, a robotic travel aid          the blind users’ canes are equipped with RFID receivers,
to guide the visually impaired on the Yamanashi University      while RFID transmitters are placed at hallway intersections.
campus. HARINOBU-6 is a motor wheel chair equipped with         As the users pass through transmitters, they hear over their
a vision system, sonars, a differential GPS, and a portable     headsets commands like turn left, turn right, and go straight.
GIS.                                                               Since RoboCart is designed to exploit structural regulari-
   RoboCart itself grew out of the Robot-Assisted Navigation    ties present in grocery stores, the overall design philosophy
project currently under way at the Assistive Technology         is similar in spirit to the research conducted by Fu et al.
Laboratory of the USU CS Department [9]. The project            [3] on action and perception in man-made environments. Fu
has developed a robotic guide for the visually impaired for     et al. [3] developed SHOPPER, a simulated robot that uses
indoor office environments. The guide, whose name is RG,         the structural and functional regularities of grocery stores to
which abbreviates robotic guide, was successfully deployed      optimize and simplify its sensing routines to shop efficiently.
and tested with visually impaired participants in two dynamic
and complex indoor environments [9], [10].                                   III. H ARDWARE AND S OFTWARE
   Insomuch as RoboCart’s localization is based on Radio           Like its predecessor, RG [9], RoboCart is built on top
Frequency Identification (RFID), RoboCart is related to sev-     of the Pioneer 2DX robotic platform from the ActivMedia
eral robotics projects that used RFID for indoor navigation.    Corporation (See Fig. 1). RoboCart’s navigation system,
Kantor and Singh used RFID tags for robot localization          called a Wayfinding Toolkit (WT), resides in a polyvinyl
and mapping [7]. Once the positions of the RFID tags are        chloride (PVC) pipe structure mounted on top of the platform.
known, their system uses time-of-arrival type of informa-       The WT includes a DellT M Ultralight X300 laptop connected
tion to estimate the distance from detected tags. Tsukiyama     to the platform’s microcontroller, a laser range finder from
[15] developed a navigation system for mobile robots using      SICK, Inc., and to a radio-frequency identification (RFID)
RFID tags. The system assumes perfect signal reception and      reader. The TI Series 2000 RFID reader is connected to a
measurement and does not deal with uncertainty. Hahnel          square 200mm × 200mm antenna. Fig. 2 shows a TI RFID
et al. [4] developed a robotic mapping and localization         Slim Disk tag attached to a grocery store shelf. These tags
system to analyze whether RFID can be used to improve the       can be attached to any objects in the environment or worn
localization of mobile robots in a simple office environment.    on clothing. They do not require any external power source
They proposed a probabilistic measurement model for RFID        or direct line of sight to be detected by the RFID reader.
readers that accurately localizes RFID tags in a simple office   The tags are activated by the spherical electromagnetic field
environment. The SpotOn system developed at the University      generated by the RFID antenna with a radius of approxi-
of Washington [5] is an RFID-based localization system for      mately 1.5 meters. Each tag is programmatically assigned
indoor environments. The system relies on the strength of the   a unique ID. When detected, RFID tags enable and disable
local navigation behaviors, e.g., follow-aisle, turn-left, turn-
right, avoid-obstacle, make-u-turn, etc.
   The basic philosophy behind this approach is to alle-
viate localization and navigation problems of completely
autonomous approaches by instrumenting environments with
inexpensive and reliable sensors that can be placed in and out
of environments without disrupting any indigenous activities.
   RoboCart’s software architecture currently includes three
components: a user interface (UI), a path planner, and a
behavior manager. The UI’s input is entered by the user
from a hand-held keypad. The original UI also had speech
input. However, speech was abandoned after unsuccessful
trials with visually impaired participants [8]. The UI’s output
mode uses non-verbal audio beacons and speech synthesis.
Destinations entered by the user from the keypad are turned
into goals for the path planner. The user can learn the
available commands from a Braille directory, a roll of paper
with Braille signs, that attaches to the handle at the back of                        Fig. 3.   Software architecture.
the robot. The directory contains a mapping of key sequences
to destinations, e.g., produce, deli, coffee shop, etc.
   The path planner and behavior manager are inspired by            update the latest sensor readings and 2) to post messages for
and partially realize Kupiers’ Spatial Semantic Hierarchy           other modules.
(SSH) [11]. The SSH is a framework for representing spa-               The behavior manager, in turn, consists of six modules:
tial knowledge. It divides spatial knowledge of autonomous          sensor module (SMod), motion control module (MCMod),
agents, e.g., humans, animals, and robots, into four levels: the    special behaviors module (SBMod), error detection and
control level, causal level, topological level, and metric level.   correction module (EMod), planning and execution module
The control level consists of low level mobility laws, e.g.,        (PEMod), and decision module (DMod).
trajectory following and aligning with a surface. The causal           The SMod is the interface to the robot’s sensory system.
level represents the world in terms of views and actions.           The SMod uses the ARIA’s API to obtain the latest sensor
A view is a collection of data items that an agent gathers          readings. The MCMod partially corresponds to the control
from its sensors. Actions move agents from view to view.            level of the SSH. This module deals with the robot’s naviga-
The topological level represents the world’s connectivity, i.e.,    tion behaviors, such as following aisles and making left, right
how different locations are connected. The metric level adds        and u-turns. The behaviors are enabled and disabled by RFID
distances between locations.                                        tags and the laser range finder’s readings. For navigation,
   The path planner realizes the causal and topological levels      the MCMod uses a potential fields approach in conjunction
of the SSH. It contains the declarative knowledge of the            with a greedy selection of empty spaces described in detail
environment and uses that knowledge to generate paths from          elsewhere [9].
point to point. The behavior manager realizes the control and          The SBMod falls into the control level of the SSH. Behav-
causal levels of the SSH. The control level is implemented          iors like entering and exiting elevators are treated as special
with the following low-level behaviors all of which run on the      behaviors since they require a different navigation sequence
WT laptop: follow-aisle, turn-left, turn-right, avoid-obstacles,    and more frequent interaction with the user. Currently, the
and make-u-turn. These behaviors are written in the behavior        SBMod implements the negotiate-elevator behavior.
programming language of the ActivMedia Robotics Interface              The EMod partially realizes the control and causal levels of
for Applications (ARIA) system from ActivMedia Robotics,            SSH. The EMod checks for errors in terms of lost paths. The
Inc.                                                                EMod relies on the RFID tags seen along the path and the
   The behavior manager also keeps track of the robot’s             behavior sequence stored in the robot’s state. If the EMod
global state. The global state is shared by all the modules. It     detects a lost path, it sends an icmd lost command to the
holds the latest sensor values, which include the laser range       DMod that causes the robot to replan and resume on user
finder readings, the lastest detected RFID tag, current veloc-       permission. It should be noted in passing that all commands
ity, current behavior state, and battery voltage. Other state       starting with icmd are internal commands, i.e., generated
parameters include: the destination, the command queue, the         by the robot. The PEMod is responsible for the stepwise
plan to reach the destination, and internal timers. The other       execution of the plan received from the planner. It keeps
modules use the current state in two ways: 1) to access and         checking if the tags detected are part of the current plan. As
soon as a valid tag is detected, the corresponding behavior
is triggered. It generates the commands, such as icmd left,
icmd right, icmd uturn, based on what tags are detected.
   The DMod partially realizes the causal level of the SSH. It
polls the command queue for new commands and triggers the
appropriate behaviors when a new command is received. It
deals with two types of commands: navigation commands,
such as “go to location X,” “pause,” “resume,” “cancel”
(cancelling a run), and information commands like “where
am I?”. The DMod is also responsible for the clean-up work
for the icmd stop and icmd lost commands. The icmd stop
command is generated when the destination tag is encoun-
tered. The icmd lost command is generated when the EMod                           Fig. 4.   A tag connectivity graph.
detects that the robot is lost.
   The UI translates all commands into an internal command
representation for the DMod. For example, if location X          can reach nodes 11 and 12. Each reachable node has a
is mapped to tag 17, the command “go to location X,”             sequence of behaviors associated with it. A single behavior
is received by the DMod as “tag:17.” As soon as this             is represented by a predicate, such as #$sFollowAisle.
command is received, the DMod obtains a plan from the            The only argument to the $sFollowAisle predicate is 0
planner and sends an icmd move command to the MCMod.             or 1 with 0 standing for the left side and 1 standing for the
On receiving the commands “pause,” “resume,” “cancel,” the       right side. In the above example, tag 11 can be reached from
DMod sends the icmd pause, icmd resume, and icmd stop            tag 10 by first making a u-turn and then following the right
commands, respectively, to the MCMod. These commands             side of the aisle until tag 11 is detected. Similarly, tag 12
make the robot change its internal behavior states.              can be reached from tag 11 by following the right side of
                                                                 the aisle in the same direction.
A. Knowledge representation                                         While OpenCyc provides a great deal of versatility and
   Knowledge engineering for the path planner is done in         power in declarative knowledge engineering, experiments
OpenCyc [18], a free knowledge engineering tool from the         with RoboCart demonstrated that the OpenCyc server cannot
Cyc Corporation that allows one to represent common sense        meet the real time commitments necessary for a deployed
knowledge using first order logic. The knowledge base repre-      robotic system. Consequently, for efficiency reasons, all
sents an aerial view of the store in which RoboCart operates.    OpenCyc representations, after they are finalized, are auto-
Currently, the knowledge base consists of tag connectivity       matically converted into a set of text files. Specifically, these
graphs, tag to destination mappings, and low-level behavior      text files contain a set of Lisp-like s-expressions that contain
scripts associated with specific tags. The environment is         the same information on the topology of a given environment.
represented as a graph where nodes represent the RFID tags       For example, (path 12 13 f) asserts that tags 12 and 13 are
and the edges represent the behaviors required to travel from    connected and that tag 13 can be reached from tag 12 by
one tag to another. Views consist of laser range finder read-     following the hallway. Thus, a grocery store is represented
ings and the IDs of the RFID tags currently detectable. For      as a connected graph similar to the one shown in Fig. 4.
example, the following assertion in the OpenCyc’s knowledge
representation language CYCL represents a graph node:            B. Plan execution
                                                                    Given the connectivity graph, the planner uses a standard
(#$rfidTag 10
                                                                 breadth first search (BFS) to find the shortest path from the
  (#$TheList
                                                                 start tag to the destination tag. The path is a tag-behavior-tag
    (#$TheList 11
                                                                 sequence. For example, if the start tag is 4 and the destination
      (#$sMakeAisleUturn)
                                                                 tag is 17, (4 l 5 f 6 r 7 f 8 f 15 f 16 r 17) is a path. The
      (#$sFollowAisle 1))
                                                                 PEMod uses this plan to guide RoboCart along the path.
    (#$TheList 12
                                                                    The behavior manager treats a plan as a sequence of
      (#$sFollowAisle 1)))
                                                                 records with three fields: prev-tag, behavior, and next-tag. In
  (#$TheList 4))
                                                                 other words, a plan is a sequence of triples < T i , Bij , Tj >,
   This assertion states that this node is represented by        where Ti and Tj are the tags connected by the behavior
the RFID tag whose ID is 10. The second argument to              Bij . If Ti and the currently detected tag are identical, the
the predicate #$rfidTag is a list of nodes that can be           current tag is valid and triggers the behavior B ij , which,
reached from node 10. In this example, from node 10 one          if successfully executed, allows the robot to detect T j , at
which point Bij is continued, disabled, or replaced with a
new behavior.
   As an example, consider the plan (1 f 2 r 3). As soon as
the valid tag 1 is detected, follow-aisle is triggered. This is
the guide’s default behavior when an icmd move command
is given. RoboCart follows the aisle until tag 2 is detected.
The detection of tag 2 is followed by the generation of the
icmd right command, which triggers turn-right. The turning
behavior is complete when shelves are detected on both sides
by the laser range finder within a specified range.

                IV. S COPE AND F EASIBILITY
   Several important issues will be beyond the project’s scope.
First, RoboCart guides its users to locations, but not inside
location. For example, RoboCart will guide its user to a
restroom and wait for her near the entrance, but will not
guide the user inside the restroom.
   Second, RoboCart is not meant for individual ownership.
The intention is not to replace the guide dog or white cane.
The intention is to help the dog’s handler or the white cane
user to overcome the limitations of their navigational aids in
grocery stores. Dynamic and complex indoor environments,                           Fig. 5.   RoboCart during a trial run.
such as grocery stores and airports, are both feasible and
socially valid for robot-assisted navigation. Guide dogs, white
canes, and other navigation devices are of limited use in           use an entirely different set of sening mechanisms, e.g., a
such environments, because they cannot help their users             glove-embedded barcode scanner or a mini-camera. In sum,
localize and find paths to useful destinations. Furthermore,         RoboCart will lead its user to the shelf that contains the
such environments can be instrumented with small sensors            needed item but will not assist the user in finding that item
that make robot-assisted navigation feasible.                       on the shelf.
   Third, it is assumed that institutions, such as grocery stores
and airports, will operate such guides on the premises in                                      V. S TATUS
the future. Estimates from the manufactures of the hardware
components that comprise the Wayfinding Toolkit indicate                RoboCart has been repeatedly deployed in Lee’s Market-
that, when produced in large quantities, the cost of the            Place, a grocery store in Logan, Utah, over a period of
toolkit may be 3,500-4,000 USD. This is an affordable               five months, from August 2004 to December 2005. Fig.
price for grocery stores and airports. For example, each            5 shows RoboCart during a trial run in the grocery store.
WalMart store already has 2-3 assisted mobility vehicles            Continuous deployment in the store was not logistically
called MartCarts(TM) to which wayfinding toolkits can be             possible. Therefore, the research team would bring the robot
attached in the future. The same modifications can be done           to the store for several hours at a time, deploy the robot
to SmartCarts(TM) available at all major airports.                  and the RFID tags, and run the robot on various routes. The
   Fourth, there is no plan to extend the domain of RoboCart        area in which the robot was tested covered a total of five
to outdoor environments. The state of the art in outdoor            aisles. The tags were placed on both sides of the aisle to
robot navigation technology does not allow one to reliably          mark different items as well as the beginning and end of
navigate outdoor environments. Additionally, the expense of         each aisle.
deploying and maintaining such systems is prohibitive to               The original design included the possibility for the user to
many organizations. Given the state of the art in outdoor           follow RoboCart holding a dog leash attached to the robot’s
localization and computer vision, outdoor environments are          battery bay. However, several visually impaired people indi-
not currently within reach of robots unless the robots are          cated that they would prefer something more static so that
teleoperated, at least part of the time.                            they could get more haptic feedback on the direction and
   Fifth, RoboCart is not intended to help its customers            motion of the robot. The dog leash was subsequently replaced
with picking specific items from shelves. Finding a specific          with a static PVC handle shown in Fig. 6. Several visually
item on a shelf is a different problem that has little to           impaired people indicated that the handle is better than the
do with navigation. A solution to that problem is likely to         dog leash.
USU Center for Persons with Disabilities, and Mr. Martin
                                                                     Blair, Director of the Utah Assistive Technology Program, for
                                                                     administrative support and assistance. The authors would like
                                                                     to thank Mr. Sachin Pavithran, a visually impaired training
                                                                     and development specialist at the USU Center for Persons
                                                                     with Disabilities, for generously volunteering his time for
                                                                     testing RoboCart.

                                                                                                 R EFERENCES
                    Fig. 6.   RoboCart’s handle.                     [1] J. Borenstein and I. Ulrich, “The GuideCane - a computerized travel
                                                                         guide for the active guidance of blind pedestrians,” IEEE International
                                                                         Conference on Robotics and Automation, 1994, San Diego, CA: IEEE
                                                                         Press.
   Instrumentating environments with RFID sensors is fast                                                       a
                                                                     [2] W. Burgard, A. Cremers, D. Fox, D. H¨hnel, G. Lakemeyer, D. Schulz,
and requires only commercial off-the-shelf (COTS) hardware               W. Steiner, and S. Thrun, “Experiences with an Interactive Museum
                                                                         Tour-Guide Robot,” Artificial Intelligence, 114:3-55, 1999.
and software components. RFID tags are inexpensive (15               [3] D. Fu, K. Hammond, and M. Swain, “Action and perception in man-
USD a piece), reliable, and easy to maintain, because they do            made environments,” 14th International Joint Conference on Artifi-
not require external power supplies. RFID provides accurate              cial Intelligence (IJCAI-95), August 1995, Montreal, Quebec, Canada:
                                                                         AAAI/MIT Press.
localization. Although RFID tag reading failures occur, they                  a
                                                                     [4] D. H¨hnel, W. Burgard, D. Fox, K. Fishkin, and M. Philipose, “Mapping
are rare and can be recovered from as long as other RFID tags            and Localization with RFID Technology,” Technical Report IRS-TR-03-
are placed in the environment. The placement of RFID tags                014, December 2003, Intel Research Institute, Seattle, WA.
                                                                     [5] J. Highthower, R. Want, and G. Borriello “SpotOn: an indoor 3D
in the environment does not seem to disrupt any indigenous               location sensing technology based on RF signal Strength,” Technical
activities. People who work in the grocery store did not seem            Report CSE 2000-02-02, 2000, University of Washington, Seattle, WA.
to mind the tags due to their small sizes.                           [6] I. Horswill, “Polly: a vision-based artificial agent,” 11th Conference
                                                                         of the American Association for Artificial Intelligence (AAAI-93),
   During the trial runs it became clear that it was difficult to         Washington, D.C.: AAAI/MIT Press.
do u-turns in store aisles. The reason for this is that it is hard   [7] G. Kantor and S. Singh, “Preliminary results in range-Only localization
for the user to hold on to the handle when the robot executes            and mapping,” IEEE Conference on Robotics and Automation, May
                                                                         2002, Washington D.C.: IEEE Press.
a u-turn. An additional complication is other shoppers in the        [8] V. Kulyukin, C. Gharpure, P. Sute, N. De Graw, J. Nicholson, and S.
same aisle that may be in the way of the handle. Lack of                 Pavithran, “A Robotic wayfinding system for the visually impaired,”
knowledge of when executing a u-turn is appropriate is a                 Innovative Applications of Artificial Intelligence (IAAI-04) Conference,
                                                                         July 2004, San Jose, CA: AAAI/MIT Press.
limitation of the system.                                            [9] V. Kulyukin, C. Gharpure, J. Nicholson, and S. Pavithran, “RFID in
   Another limitation is that the spacial sensing of the system          robot-assisted indoor navigation for the visually impaired,” IEEE/RSJ
is confined to a horizontal plane 180 degrees in front of the             Intelligent Robots and Systems (IROS 2004) Conference, September -
                                                                         October 2004, Sendai, Japan: Sendai Kyodo Printing Co.
robot and approximately 50 centimeters from the floor, i.e.,          [10] V. Kulyukin, C. Gharpure, N. De Graw, J. Nicholson, S. Pavithran,
the signal plane of the robot’s laser range finder. This allows           “A Robotic guide for the visually impaired in indoor environments,”
the robot to avoid shoppers and boxes in the aisles. However,            Rehabilitation Engineering and Assistive Technology Society of North
                                                                         America (RESNA 2004) Conference, June 2004, Orland, FL: Avail. on
it makes it impossible for the robot to detect advertisement             CD-ROM.
boards placed above 50 centimeters and attached to the               [11] B. Kupiers, “The Spatial Semantic Hierarchy,” Artificial Intelligence ,
shelves. Thus, it is possible for the basket mounted on                  119:191-233, 2000.
                                                                     [12] H. Mori and S. Kotani, “Robotic Travel Aid for the Blind:
the robot to collide with a solid object placed above the                HARUNOBU-6,” Second European Conference on Disability, Virtual
horizontal plane of the robot’s laser range finder. Additional                                                         o
                                                                         Reality, and Assistive Technology, 1998, S¨vde, Sweden.
range sensors are needed that will scan the immediate space          [13] D.A. Ross, “Implementing assistive technology on wearable comput-
                                                                         ers,” IEEE Intelligent Systems, May-June:2-8, 2001.
upward.                                                              [14] D.A. Ross and B.B. Blasch. “Development of a wearable computer
   Finally, no systematic evaluations of the system with visu-           orientation system,” IEEE Personal and Ubiquitous Computing, 6:49-
ally impaired participants have so far been conducted. Such              63, 2002.
                                                                     [15] T. Tsukiyama, “Navigation System for the Mobile Robots using
evaluations are planned in the future. These evaluations will            RFID Tags,” IEEE Conference on Advanced Robotics, June-July 2003,
undoubtedly lead to modifications of the current hardware                 Coimbra, Portugal.
and software solutions.                                              [16] S. Thrun, M. Bennewitz, W. Burgard, A. Cremers, F. Dellaert, D. Fox,
                                                                               a
                                                                         D. H¨hnel, C. Rosenberg, C. N. Roby, J. Schutle, and D. Schultz,
                                                                         “Minerva: A Second Generation Mobile Tour-Guide Robot,” IEEE
                    ACKNOWLEDGMENTS                                      International Conference on Robotics and Automation (ICRA-99), June
   The authors would like to thank Mr. Lee Badger, the owner             1999, Antwerp, Belgium.
                                                                     [17] http://www.haptica.com/walker.html, Guido, Haptica
of Lee’s MarketPlace, a grocery store in Logan, Utah, for                Corporation.
allowing them to use his store for testing RoboCart. The             [18] http://www.opencyc.org, The OpenCyc Project: Formalized
authors would like to thank Dr. Sarah Rule, Director of the              Common Knowledge, Cycorp, Inc.

Contenu connexe

Similaire à RoboCart: Toward Robot-Assisted Navigation of Grocery Stores by the Visually Impaired

RFID in Robot-Assisted Indoor Navigation for the Visually Impaired
RFID in Robot-Assisted Indoor Navigation for the Visually ImpairedRFID in Robot-Assisted Indoor Navigation for the Visually Impaired
RFID in Robot-Assisted Indoor Navigation for the Visually ImpairedVladimir Kulyukin
 
Robots as Interfaces to Haptic and Locomotor Spaces
Robots as Interfaces to Haptic and Locomotor SpacesRobots as Interfaces to Haptic and Locomotor Spaces
Robots as Interfaces to Haptic and Locomotor SpacesVladimir Kulyukin
 
Teleassistance in Accessible Shopping for the Blind
Teleassistance in Accessible Shopping for the BlindTeleassistance in Accessible Shopping for the Blind
Teleassistance in Accessible Shopping for the BlindVladimir Kulyukin
 
Paper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdf
Paper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdfPaper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdf
Paper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdfSathishRadcliffe
 
IRJET- Indoor Shopping System for Visually Impaired People
IRJET- Indoor Shopping System for Visually Impaired PeopleIRJET- Indoor Shopping System for Visually Impaired People
IRJET- Indoor Shopping System for Visually Impaired PeopleIRJET Journal
 
Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...
Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...
Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...Vladimir Kulyukin
 
Mobile device self-location method using at least one passive radio-frequenc...
Mobile device  self-location method using at least one passive radio-frequenc...Mobile device  self-location method using at least one passive radio-frequenc...
Mobile device self-location method using at least one passive radio-frequenc...Toscana Open Research
 
DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...
DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...
DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...VLSICS Design
 
IRJET- Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...
IRJET-  	  Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...IRJET-  	  Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...
IRJET- Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...IRJET Journal
 
Android based autonomous coloured line follower robot
Android based autonomous coloured line follower robotAndroid based autonomous coloured line follower robot
Android based autonomous coloured line follower roboteSAT Publishing House
 
Robotics as Embedded systems
Robotics as Embedded systems Robotics as Embedded systems
Robotics as Embedded systems HayderThary
 
Robotics in Medical Science
Robotics in Medical ScienceRobotics in Medical Science
Robotics in Medical ScienceVashuGupta8
 
Wireless Pick & Place Robot
Wireless Pick & Place RobotWireless Pick & Place Robot
Wireless Pick & Place RobotMarmik Kothari
 
State of robotics in logistics
State of robotics in logisticsState of robotics in logistics
State of robotics in logisticsshasanka sahu
 

Similaire à RoboCart: Toward Robot-Assisted Navigation of Grocery Stores by the Visually Impaired (20)

RFID in Robot-Assisted Indoor Navigation for the Visually Impaired
RFID in Robot-Assisted Indoor Navigation for the Visually ImpairedRFID in Robot-Assisted Indoor Navigation for the Visually Impaired
RFID in Robot-Assisted Indoor Navigation for the Visually Impaired
 
Robots as Interfaces to Haptic and Locomotor Spaces
Robots as Interfaces to Haptic and Locomotor SpacesRobots as Interfaces to Haptic and Locomotor Spaces
Robots as Interfaces to Haptic and Locomotor Spaces
 
Ts 2 b topic
Ts 2 b topicTs 2 b topic
Ts 2 b topic
 
Teleassistance in Accessible Shopping for the Blind
Teleassistance in Accessible Shopping for the BlindTeleassistance in Accessible Shopping for the Blind
Teleassistance in Accessible Shopping for the Blind
 
Paper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdf
Paper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdfPaper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdf
Paper_39-SRAVIP_Smart_Robot_Assistant_for_Visually_Impaired.pdf
 
Robotics power presentation.
Robotics power presentation.Robotics power presentation.
Robotics power presentation.
 
IRJET- Indoor Shopping System for Visually Impaired People
IRJET- Indoor Shopping System for Visually Impaired PeopleIRJET- Indoor Shopping System for Visually Impaired People
IRJET- Indoor Shopping System for Visually Impaired People
 
Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...
Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...
Eyes-Free Barcode Localization and Decoding for Visually Impaired Mobile Phon...
 
Ijmet 10 01_168
Ijmet 10 01_168Ijmet 10 01_168
Ijmet 10 01_168
 
Mobile device self-location method using at least one passive radio-frequenc...
Mobile device  self-location method using at least one passive radio-frequenc...Mobile device  self-location method using at least one passive radio-frequenc...
Mobile device self-location method using at least one passive radio-frequenc...
 
DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...
DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...
DESIGN AND VLSIIMPLEMENTATION OF ANTICOLLISION ENABLED ROBOT PROCESSOR USING ...
 
final_report
final_reportfinal_report
final_report
 
IRJET- Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...
IRJET-  	  Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...IRJET-  	  Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...
IRJET- Design, Manufacturing and Analysis of Robotic Arm with SCARA Confi...
 
Android based autonomous coloured line follower robot
Android based autonomous coloured line follower robotAndroid based autonomous coloured line follower robot
Android based autonomous coloured line follower robot
 
Robotics as Embedded systems
Robotics as Embedded systems Robotics as Embedded systems
Robotics as Embedded systems
 
Robotics in Medical Science
Robotics in Medical ScienceRobotics in Medical Science
Robotics in Medical Science
 
Roboclub, IITK (2008)
Roboclub, IITK (2008)Roboclub, IITK (2008)
Roboclub, IITK (2008)
 
Wireless Pick & Place Robot
Wireless Pick & Place RobotWireless Pick & Place Robot
Wireless Pick & Place Robot
 
A010430107
A010430107A010430107
A010430107
 
State of robotics in logistics
State of robotics in logisticsState of robotics in logistics
State of robotics in logistics
 

Plus de Vladimir Kulyukin

Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...
Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...
Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...Vladimir Kulyukin
 
Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...
Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...
Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...Vladimir Kulyukin
 
Generalized Hamming Distance
Generalized Hamming DistanceGeneralized Hamming Distance
Generalized Hamming DistanceVladimir Kulyukin
 
Adapting Measures of Clumping Strength to Assess Term-Term Similarity
Adapting Measures of Clumping Strength to Assess Term-Term SimilarityAdapting Measures of Clumping Strength to Assess Term-Term Similarity
Adapting Measures of Clumping Strength to Assess Term-Term SimilarityVladimir Kulyukin
 
A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...
A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...
A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...Vladimir Kulyukin
 
Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...
Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...
Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...Vladimir Kulyukin
 
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...Vladimir Kulyukin
 
Text Skew Angle Detection in Vision-Based Scanning of Nutrition Labels
Text Skew Angle Detection in Vision-Based Scanning of Nutrition LabelsText Skew Angle Detection in Vision-Based Scanning of Nutrition Labels
Text Skew Angle Detection in Vision-Based Scanning of Nutrition LabelsVladimir Kulyukin
 
Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...
Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...
Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...Vladimir Kulyukin
 
Effective Nutrition Label Use on Smartphones
Effective Nutrition Label Use on SmartphonesEffective Nutrition Label Use on Smartphones
Effective Nutrition Label Use on SmartphonesVladimir Kulyukin
 
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...Vladimir Kulyukin
 
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the Cloud
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the CloudAn Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the Cloud
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the CloudVladimir Kulyukin
 
Narrative Map Augmentation with Automated Landmark Extraction and Path Inference
Narrative Map Augmentation with Automated Landmark Extraction and Path InferenceNarrative Map Augmentation with Automated Landmark Extraction and Path Inference
Narrative Map Augmentation with Automated Landmark Extraction and Path InferenceVladimir Kulyukin
 
Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...
Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...
Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...Vladimir Kulyukin
 
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...Vladimir Kulyukin
 
Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...
Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...
Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...Vladimir Kulyukin
 
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...Vladimir Kulyukin
 
Toward Blind Travel Support through Verbal Route Directions: A Path Inference...
Toward Blind Travel Support through Verbal Route Directions: A Path Inference...Toward Blind Travel Support through Verbal Route Directions: A Path Inference...
Toward Blind Travel Support through Verbal Route Directions: A Path Inference...Vladimir Kulyukin
 
Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...
Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...
Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...Vladimir Kulyukin
 
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...Vladimir Kulyukin
 

Plus de Vladimir Kulyukin (20)

Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...
Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...
Toward Sustainable Electronic Beehive Monitoring: Algorithms for Omnidirectio...
 
Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...
Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...
Digitizing Buzzing Signals into A440 Piano Note Sequences and Estimating Fora...
 
Generalized Hamming Distance
Generalized Hamming DistanceGeneralized Hamming Distance
Generalized Hamming Distance
 
Adapting Measures of Clumping Strength to Assess Term-Term Similarity
Adapting Measures of Clumping Strength to Assess Term-Term SimilarityAdapting Measures of Clumping Strength to Assess Term-Term Similarity
Adapting Measures of Clumping Strength to Assess Term-Term Similarity
 
A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...
A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...
A Cloud-Based Infrastructure for Caloric Intake Estimation from Pre-Meal Vide...
 
Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...
Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...
Exploring Finite State Automata with Junun Robots: A Case Study in Computabil...
 
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...
Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed ...
 
Text Skew Angle Detection in Vision-Based Scanning of Nutrition Labels
Text Skew Angle Detection in Vision-Based Scanning of Nutrition LabelsText Skew Angle Detection in Vision-Based Scanning of Nutrition Labels
Text Skew Angle Detection in Vision-Based Scanning of Nutrition Labels
 
Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...
Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...
Vision-Based Localization and Scanning of 1D UPC and EAN Barcodes with Relaxe...
 
Effective Nutrition Label Use on Smartphones
Effective Nutrition Label Use on SmartphonesEffective Nutrition Label Use on Smartphones
Effective Nutrition Label Use on Smartphones
 
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...
An Algorithm for Mobile Vision-Based Localization of Skewed Nutrition Labels ...
 
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the Cloud
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the CloudAn Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the Cloud
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning in the Cloud
 
Narrative Map Augmentation with Automated Landmark Extraction and Path Inference
Narrative Map Augmentation with Automated Landmark Extraction and Path InferenceNarrative Map Augmentation with Automated Landmark Extraction and Path Inference
Narrative Map Augmentation with Automated Landmark Extraction and Path Inference
 
Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...
Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...
Skip Trie Matching: A Greedy Algorithm for Real-Time OCR Error Correction on ...
 
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
 
Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...
Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...
Skip Trie Matching for Real Time OCR Output Error Correction on Android Smart...
 
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
Vision-Based Localization & Text Chunking of Nutrition Fact Tables on Android...
 
Toward Blind Travel Support through Verbal Route Directions: A Path Inference...
Toward Blind Travel Support through Verbal Route Directions: A Path Inference...Toward Blind Travel Support through Verbal Route Directions: A Path Inference...
Toward Blind Travel Support through Verbal Route Directions: A Path Inference...
 
Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...
Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...
Eye-Free Barcode Detection on Smartphones with Niblack's Binarization and Sup...
 
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...
Eyesight Sharing in Blind Grocery Shopping: Remote P2P Caregiving through Clo...
 

Dernier

Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 

Dernier (20)

Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 

RoboCart: Toward Robot-Assisted Navigation of Grocery Stores by the Visually Impaired

  • 1. RoboCart: Toward Robot-Assisted Navigation of Grocery Stores by the Visually Impaired∗ Vladimir Kulyukin Chaitanya Gharpure John Nicholson Computer Science Assistive Technology Laboratory Department of Computer Science Utah State University Logan, Utah, U.S.A {vkulyukin, cpg, jnicholson}@cc.usu.edu Abstract— This paper presents RoboCart, a proof-of-concept prototype of a robotic shopping assistant for the visually impaired. The purpose of RoboCart is to help visually impaired customers navigate a typical grocery store and carry purchased items. The hardware and software components of the system are presented. For localization, RoboCart relies on RFID tags deployed at various locations in the store. For navigation, Robo- Cart relies on laser range finding. Experiences with deploying RoboCart in a real grocery store are described. The current status of the system and its limitations are outlined. Index Terms— service robotics I. I NTRODUCTION Grocery shopping is a routine activity that people all over the world perform on a regular basis. However, grocery stores and supermarkets remain largely inaccessible to people with visual impairments. The main barrier is the inadequacy of the principal navigation aids, such as guide dogs and white canes, for negotiating the seemingly simple topological structure of a typical grocery store. While guide dogs are helpful in Fig. 1. RoboCart in a grocery store. micro-navigation, e.g., local obstacle avoidance and homing in on simple targets such as exit doors, they offer little assistance with macro-navigation, which requires topological the grocery shopping for the visually impaired. Rather, the knowledge of the environment. A guide dog may pick up purpose is to assist visually impaired customers in navigating and remember routes to a small set of grocery items through the store and carrying the purchased items around the store repeated exposure to those routes. However, the guide dog and to the check-out registers. cannot help its handler in a situation where the store changes This paper describes RoboCart, a proof-of-concept proto- the location of several items between visits or stops carrying type of a robotic grocery shopping assistant for the visually an item. Nor can the dog assist its handler with pushing a impaired. RoboCart is shown in Fig. 1. The paper is orga- shopping cart. Needless to say, the white cane does not fare nized as follows. First, related work is reviewed. Second, any better under the same circumstances. RoboCart’s software and hardware components are presented. In August 2004, the Assistive Technology Laboratory of Third, the scope and feasibility issues are discussed. Finally, the Department of Computer Science (CS) of Utah State the current status of the system is described through experi- University (USU) and the USU Center for Persons with ences with deploying RoboCart in a real grocery store. Disabilities (CPD) started a collaborative project whose ob- jective is to build a robotic shopping assistant for the visually II. R ELATED W ORK impaired. The purpose of the shopping assistant is not to do Several researchers have developed robotic guides for ∗ This work is partially supported by NSF Grant IIS-0346880 and two indoor environments. While these systems are not explicitly Community University Research Initiative (CURI) grants (CURI 2004 and designed for the visually impaired, they offer valuable fea- CURI 2005) from the State of Utah. All grants are awarded to V. Kulyukin. sibility studies in robotic navigation of indoor environments.
  • 2. Horswill [6] built Polly, a small robotic guide for the MIT AI Lab. Polly used lightweight vision routines that depended on textures specific to the lab. Burgard et al. [2] developed RHINO, an interactive tour guide that was deployed in a museum in Germany for a period of six days. Thrun et al. [16] used the approaches developed in RHINO to build MINERVA, an autonomous tour-guide robot that was deployed in the National Museum of American History in Washington, D.C. Borenstein and Ulrich introduced GuideCane [7], a mobile obstacle avoidance device for the visually impaired. Guide- Cane consists of a long handle and a sensor unit mounted on a steerable two-wheel axle. The sensor unit consists of ultrasonic sensors that detect obstacles and help the user to steer the device around them. A steering servo motor, operating under the control of the onboard computer, steers Fig. 2. RFID tag in a grocery store. the wheels relative to the handle. The Haptica Corporation developed Guido, a robotic walk- ing frame for people with impaired vision and reduced signal from RF tags and uses triangulation to estimate their mobility [17]. Guido uses the onboard sonars to scan the positions. Another RFID-based navigation system for indoor immediate environment for obstacles and communicates de- environments was developed at the Atlanta VA Rehabilitation tected obstacles to the user via speech synthesis. Mori and Research and Development Center [13], [14]. In this system, Kotani [12] developed HARINOBU-6, a robotic travel aid the blind users’ canes are equipped with RFID receivers, to guide the visually impaired on the Yamanashi University while RFID transmitters are placed at hallway intersections. campus. HARINOBU-6 is a motor wheel chair equipped with As the users pass through transmitters, they hear over their a vision system, sonars, a differential GPS, and a portable headsets commands like turn left, turn right, and go straight. GIS. Since RoboCart is designed to exploit structural regulari- RoboCart itself grew out of the Robot-Assisted Navigation ties present in grocery stores, the overall design philosophy project currently under way at the Assistive Technology is similar in spirit to the research conducted by Fu et al. Laboratory of the USU CS Department [9]. The project [3] on action and perception in man-made environments. Fu has developed a robotic guide for the visually impaired for et al. [3] developed SHOPPER, a simulated robot that uses indoor office environments. The guide, whose name is RG, the structural and functional regularities of grocery stores to which abbreviates robotic guide, was successfully deployed optimize and simplify its sensing routines to shop efficiently. and tested with visually impaired participants in two dynamic and complex indoor environments [9], [10]. III. H ARDWARE AND S OFTWARE Insomuch as RoboCart’s localization is based on Radio Like its predecessor, RG [9], RoboCart is built on top Frequency Identification (RFID), RoboCart is related to sev- of the Pioneer 2DX robotic platform from the ActivMedia eral robotics projects that used RFID for indoor navigation. Corporation (See Fig. 1). RoboCart’s navigation system, Kantor and Singh used RFID tags for robot localization called a Wayfinding Toolkit (WT), resides in a polyvinyl and mapping [7]. Once the positions of the RFID tags are chloride (PVC) pipe structure mounted on top of the platform. known, their system uses time-of-arrival type of informa- The WT includes a DellT M Ultralight X300 laptop connected tion to estimate the distance from detected tags. Tsukiyama to the platform’s microcontroller, a laser range finder from [15] developed a navigation system for mobile robots using SICK, Inc., and to a radio-frequency identification (RFID) RFID tags. The system assumes perfect signal reception and reader. The TI Series 2000 RFID reader is connected to a measurement and does not deal with uncertainty. Hahnel square 200mm × 200mm antenna. Fig. 2 shows a TI RFID et al. [4] developed a robotic mapping and localization Slim Disk tag attached to a grocery store shelf. These tags system to analyze whether RFID can be used to improve the can be attached to any objects in the environment or worn localization of mobile robots in a simple office environment. on clothing. They do not require any external power source They proposed a probabilistic measurement model for RFID or direct line of sight to be detected by the RFID reader. readers that accurately localizes RFID tags in a simple office The tags are activated by the spherical electromagnetic field environment. The SpotOn system developed at the University generated by the RFID antenna with a radius of approxi- of Washington [5] is an RFID-based localization system for mately 1.5 meters. Each tag is programmatically assigned indoor environments. The system relies on the strength of the a unique ID. When detected, RFID tags enable and disable
  • 3. local navigation behaviors, e.g., follow-aisle, turn-left, turn- right, avoid-obstacle, make-u-turn, etc. The basic philosophy behind this approach is to alle- viate localization and navigation problems of completely autonomous approaches by instrumenting environments with inexpensive and reliable sensors that can be placed in and out of environments without disrupting any indigenous activities. RoboCart’s software architecture currently includes three components: a user interface (UI), a path planner, and a behavior manager. The UI’s input is entered by the user from a hand-held keypad. The original UI also had speech input. However, speech was abandoned after unsuccessful trials with visually impaired participants [8]. The UI’s output mode uses non-verbal audio beacons and speech synthesis. Destinations entered by the user from the keypad are turned into goals for the path planner. The user can learn the available commands from a Braille directory, a roll of paper with Braille signs, that attaches to the handle at the back of Fig. 3. Software architecture. the robot. The directory contains a mapping of key sequences to destinations, e.g., produce, deli, coffee shop, etc. The path planner and behavior manager are inspired by update the latest sensor readings and 2) to post messages for and partially realize Kupiers’ Spatial Semantic Hierarchy other modules. (SSH) [11]. The SSH is a framework for representing spa- The behavior manager, in turn, consists of six modules: tial knowledge. It divides spatial knowledge of autonomous sensor module (SMod), motion control module (MCMod), agents, e.g., humans, animals, and robots, into four levels: the special behaviors module (SBMod), error detection and control level, causal level, topological level, and metric level. correction module (EMod), planning and execution module The control level consists of low level mobility laws, e.g., (PEMod), and decision module (DMod). trajectory following and aligning with a surface. The causal The SMod is the interface to the robot’s sensory system. level represents the world in terms of views and actions. The SMod uses the ARIA’s API to obtain the latest sensor A view is a collection of data items that an agent gathers readings. The MCMod partially corresponds to the control from its sensors. Actions move agents from view to view. level of the SSH. This module deals with the robot’s naviga- The topological level represents the world’s connectivity, i.e., tion behaviors, such as following aisles and making left, right how different locations are connected. The metric level adds and u-turns. The behaviors are enabled and disabled by RFID distances between locations. tags and the laser range finder’s readings. For navigation, The path planner realizes the causal and topological levels the MCMod uses a potential fields approach in conjunction of the SSH. It contains the declarative knowledge of the with a greedy selection of empty spaces described in detail environment and uses that knowledge to generate paths from elsewhere [9]. point to point. The behavior manager realizes the control and The SBMod falls into the control level of the SSH. Behav- causal levels of the SSH. The control level is implemented iors like entering and exiting elevators are treated as special with the following low-level behaviors all of which run on the behaviors since they require a different navigation sequence WT laptop: follow-aisle, turn-left, turn-right, avoid-obstacles, and more frequent interaction with the user. Currently, the and make-u-turn. These behaviors are written in the behavior SBMod implements the negotiate-elevator behavior. programming language of the ActivMedia Robotics Interface The EMod partially realizes the control and causal levels of for Applications (ARIA) system from ActivMedia Robotics, SSH. The EMod checks for errors in terms of lost paths. The Inc. EMod relies on the RFID tags seen along the path and the The behavior manager also keeps track of the robot’s behavior sequence stored in the robot’s state. If the EMod global state. The global state is shared by all the modules. It detects a lost path, it sends an icmd lost command to the holds the latest sensor values, which include the laser range DMod that causes the robot to replan and resume on user finder readings, the lastest detected RFID tag, current veloc- permission. It should be noted in passing that all commands ity, current behavior state, and battery voltage. Other state starting with icmd are internal commands, i.e., generated parameters include: the destination, the command queue, the by the robot. The PEMod is responsible for the stepwise plan to reach the destination, and internal timers. The other execution of the plan received from the planner. It keeps modules use the current state in two ways: 1) to access and checking if the tags detected are part of the current plan. As
  • 4. soon as a valid tag is detected, the corresponding behavior is triggered. It generates the commands, such as icmd left, icmd right, icmd uturn, based on what tags are detected. The DMod partially realizes the causal level of the SSH. It polls the command queue for new commands and triggers the appropriate behaviors when a new command is received. It deals with two types of commands: navigation commands, such as “go to location X,” “pause,” “resume,” “cancel” (cancelling a run), and information commands like “where am I?”. The DMod is also responsible for the clean-up work for the icmd stop and icmd lost commands. The icmd stop command is generated when the destination tag is encoun- tered. The icmd lost command is generated when the EMod Fig. 4. A tag connectivity graph. detects that the robot is lost. The UI translates all commands into an internal command representation for the DMod. For example, if location X can reach nodes 11 and 12. Each reachable node has a is mapped to tag 17, the command “go to location X,” sequence of behaviors associated with it. A single behavior is received by the DMod as “tag:17.” As soon as this is represented by a predicate, such as #$sFollowAisle. command is received, the DMod obtains a plan from the The only argument to the $sFollowAisle predicate is 0 planner and sends an icmd move command to the MCMod. or 1 with 0 standing for the left side and 1 standing for the On receiving the commands “pause,” “resume,” “cancel,” the right side. In the above example, tag 11 can be reached from DMod sends the icmd pause, icmd resume, and icmd stop tag 10 by first making a u-turn and then following the right commands, respectively, to the MCMod. These commands side of the aisle until tag 11 is detected. Similarly, tag 12 make the robot change its internal behavior states. can be reached from tag 11 by following the right side of the aisle in the same direction. A. Knowledge representation While OpenCyc provides a great deal of versatility and Knowledge engineering for the path planner is done in power in declarative knowledge engineering, experiments OpenCyc [18], a free knowledge engineering tool from the with RoboCart demonstrated that the OpenCyc server cannot Cyc Corporation that allows one to represent common sense meet the real time commitments necessary for a deployed knowledge using first order logic. The knowledge base repre- robotic system. Consequently, for efficiency reasons, all sents an aerial view of the store in which RoboCart operates. OpenCyc representations, after they are finalized, are auto- Currently, the knowledge base consists of tag connectivity matically converted into a set of text files. Specifically, these graphs, tag to destination mappings, and low-level behavior text files contain a set of Lisp-like s-expressions that contain scripts associated with specific tags. The environment is the same information on the topology of a given environment. represented as a graph where nodes represent the RFID tags For example, (path 12 13 f) asserts that tags 12 and 13 are and the edges represent the behaviors required to travel from connected and that tag 13 can be reached from tag 12 by one tag to another. Views consist of laser range finder read- following the hallway. Thus, a grocery store is represented ings and the IDs of the RFID tags currently detectable. For as a connected graph similar to the one shown in Fig. 4. example, the following assertion in the OpenCyc’s knowledge representation language CYCL represents a graph node: B. Plan execution Given the connectivity graph, the planner uses a standard (#$rfidTag 10 breadth first search (BFS) to find the shortest path from the (#$TheList start tag to the destination tag. The path is a tag-behavior-tag (#$TheList 11 sequence. For example, if the start tag is 4 and the destination (#$sMakeAisleUturn) tag is 17, (4 l 5 f 6 r 7 f 8 f 15 f 16 r 17) is a path. The (#$sFollowAisle 1)) PEMod uses this plan to guide RoboCart along the path. (#$TheList 12 The behavior manager treats a plan as a sequence of (#$sFollowAisle 1))) records with three fields: prev-tag, behavior, and next-tag. In (#$TheList 4)) other words, a plan is a sequence of triples < T i , Bij , Tj >, This assertion states that this node is represented by where Ti and Tj are the tags connected by the behavior the RFID tag whose ID is 10. The second argument to Bij . If Ti and the currently detected tag are identical, the the predicate #$rfidTag is a list of nodes that can be current tag is valid and triggers the behavior B ij , which, reached from node 10. In this example, from node 10 one if successfully executed, allows the robot to detect T j , at
  • 5. which point Bij is continued, disabled, or replaced with a new behavior. As an example, consider the plan (1 f 2 r 3). As soon as the valid tag 1 is detected, follow-aisle is triggered. This is the guide’s default behavior when an icmd move command is given. RoboCart follows the aisle until tag 2 is detected. The detection of tag 2 is followed by the generation of the icmd right command, which triggers turn-right. The turning behavior is complete when shelves are detected on both sides by the laser range finder within a specified range. IV. S COPE AND F EASIBILITY Several important issues will be beyond the project’s scope. First, RoboCart guides its users to locations, but not inside location. For example, RoboCart will guide its user to a restroom and wait for her near the entrance, but will not guide the user inside the restroom. Second, RoboCart is not meant for individual ownership. The intention is not to replace the guide dog or white cane. The intention is to help the dog’s handler or the white cane user to overcome the limitations of their navigational aids in grocery stores. Dynamic and complex indoor environments, Fig. 5. RoboCart during a trial run. such as grocery stores and airports, are both feasible and socially valid for robot-assisted navigation. Guide dogs, white canes, and other navigation devices are of limited use in use an entirely different set of sening mechanisms, e.g., a such environments, because they cannot help their users glove-embedded barcode scanner or a mini-camera. In sum, localize and find paths to useful destinations. Furthermore, RoboCart will lead its user to the shelf that contains the such environments can be instrumented with small sensors needed item but will not assist the user in finding that item that make robot-assisted navigation feasible. on the shelf. Third, it is assumed that institutions, such as grocery stores and airports, will operate such guides on the premises in V. S TATUS the future. Estimates from the manufactures of the hardware components that comprise the Wayfinding Toolkit indicate RoboCart has been repeatedly deployed in Lee’s Market- that, when produced in large quantities, the cost of the Place, a grocery store in Logan, Utah, over a period of toolkit may be 3,500-4,000 USD. This is an affordable five months, from August 2004 to December 2005. Fig. price for grocery stores and airports. For example, each 5 shows RoboCart during a trial run in the grocery store. WalMart store already has 2-3 assisted mobility vehicles Continuous deployment in the store was not logistically called MartCarts(TM) to which wayfinding toolkits can be possible. Therefore, the research team would bring the robot attached in the future. The same modifications can be done to the store for several hours at a time, deploy the robot to SmartCarts(TM) available at all major airports. and the RFID tags, and run the robot on various routes. The Fourth, there is no plan to extend the domain of RoboCart area in which the robot was tested covered a total of five to outdoor environments. The state of the art in outdoor aisles. The tags were placed on both sides of the aisle to robot navigation technology does not allow one to reliably mark different items as well as the beginning and end of navigate outdoor environments. Additionally, the expense of each aisle. deploying and maintaining such systems is prohibitive to The original design included the possibility for the user to many organizations. Given the state of the art in outdoor follow RoboCart holding a dog leash attached to the robot’s localization and computer vision, outdoor environments are battery bay. However, several visually impaired people indi- not currently within reach of robots unless the robots are cated that they would prefer something more static so that teleoperated, at least part of the time. they could get more haptic feedback on the direction and Fifth, RoboCart is not intended to help its customers motion of the robot. The dog leash was subsequently replaced with picking specific items from shelves. Finding a specific with a static PVC handle shown in Fig. 6. Several visually item on a shelf is a different problem that has little to impaired people indicated that the handle is better than the do with navigation. A solution to that problem is likely to dog leash.
  • 6. USU Center for Persons with Disabilities, and Mr. Martin Blair, Director of the Utah Assistive Technology Program, for administrative support and assistance. The authors would like to thank Mr. Sachin Pavithran, a visually impaired training and development specialist at the USU Center for Persons with Disabilities, for generously volunteering his time for testing RoboCart. R EFERENCES Fig. 6. RoboCart’s handle. [1] J. Borenstein and I. Ulrich, “The GuideCane - a computerized travel guide for the active guidance of blind pedestrians,” IEEE International Conference on Robotics and Automation, 1994, San Diego, CA: IEEE Press. Instrumentating environments with RFID sensors is fast a [2] W. Burgard, A. Cremers, D. Fox, D. H¨hnel, G. Lakemeyer, D. Schulz, and requires only commercial off-the-shelf (COTS) hardware W. Steiner, and S. Thrun, “Experiences with an Interactive Museum Tour-Guide Robot,” Artificial Intelligence, 114:3-55, 1999. and software components. RFID tags are inexpensive (15 [3] D. Fu, K. Hammond, and M. Swain, “Action and perception in man- USD a piece), reliable, and easy to maintain, because they do made environments,” 14th International Joint Conference on Artifi- not require external power supplies. RFID provides accurate cial Intelligence (IJCAI-95), August 1995, Montreal, Quebec, Canada: AAAI/MIT Press. localization. Although RFID tag reading failures occur, they a [4] D. H¨hnel, W. Burgard, D. Fox, K. Fishkin, and M. Philipose, “Mapping are rare and can be recovered from as long as other RFID tags and Localization with RFID Technology,” Technical Report IRS-TR-03- are placed in the environment. The placement of RFID tags 014, December 2003, Intel Research Institute, Seattle, WA. [5] J. Highthower, R. Want, and G. Borriello “SpotOn: an indoor 3D in the environment does not seem to disrupt any indigenous location sensing technology based on RF signal Strength,” Technical activities. People who work in the grocery store did not seem Report CSE 2000-02-02, 2000, University of Washington, Seattle, WA. to mind the tags due to their small sizes. [6] I. Horswill, “Polly: a vision-based artificial agent,” 11th Conference of the American Association for Artificial Intelligence (AAAI-93), During the trial runs it became clear that it was difficult to Washington, D.C.: AAAI/MIT Press. do u-turns in store aisles. The reason for this is that it is hard [7] G. Kantor and S. Singh, “Preliminary results in range-Only localization for the user to hold on to the handle when the robot executes and mapping,” IEEE Conference on Robotics and Automation, May 2002, Washington D.C.: IEEE Press. a u-turn. An additional complication is other shoppers in the [8] V. Kulyukin, C. Gharpure, P. Sute, N. De Graw, J. Nicholson, and S. same aisle that may be in the way of the handle. Lack of Pavithran, “A Robotic wayfinding system for the visually impaired,” knowledge of when executing a u-turn is appropriate is a Innovative Applications of Artificial Intelligence (IAAI-04) Conference, July 2004, San Jose, CA: AAAI/MIT Press. limitation of the system. [9] V. Kulyukin, C. Gharpure, J. Nicholson, and S. Pavithran, “RFID in Another limitation is that the spacial sensing of the system robot-assisted indoor navigation for the visually impaired,” IEEE/RSJ is confined to a horizontal plane 180 degrees in front of the Intelligent Robots and Systems (IROS 2004) Conference, September - October 2004, Sendai, Japan: Sendai Kyodo Printing Co. robot and approximately 50 centimeters from the floor, i.e., [10] V. Kulyukin, C. Gharpure, N. De Graw, J. Nicholson, S. Pavithran, the signal plane of the robot’s laser range finder. This allows “A Robotic guide for the visually impaired in indoor environments,” the robot to avoid shoppers and boxes in the aisles. However, Rehabilitation Engineering and Assistive Technology Society of North America (RESNA 2004) Conference, June 2004, Orland, FL: Avail. on it makes it impossible for the robot to detect advertisement CD-ROM. boards placed above 50 centimeters and attached to the [11] B. Kupiers, “The Spatial Semantic Hierarchy,” Artificial Intelligence , shelves. Thus, it is possible for the basket mounted on 119:191-233, 2000. [12] H. Mori and S. Kotani, “Robotic Travel Aid for the Blind: the robot to collide with a solid object placed above the HARUNOBU-6,” Second European Conference on Disability, Virtual horizontal plane of the robot’s laser range finder. Additional o Reality, and Assistive Technology, 1998, S¨vde, Sweden. range sensors are needed that will scan the immediate space [13] D.A. Ross, “Implementing assistive technology on wearable comput- ers,” IEEE Intelligent Systems, May-June:2-8, 2001. upward. [14] D.A. Ross and B.B. Blasch. “Development of a wearable computer Finally, no systematic evaluations of the system with visu- orientation system,” IEEE Personal and Ubiquitous Computing, 6:49- ally impaired participants have so far been conducted. Such 63, 2002. [15] T. Tsukiyama, “Navigation System for the Mobile Robots using evaluations are planned in the future. These evaluations will RFID Tags,” IEEE Conference on Advanced Robotics, June-July 2003, undoubtedly lead to modifications of the current hardware Coimbra, Portugal. and software solutions. [16] S. Thrun, M. Bennewitz, W. Burgard, A. Cremers, F. Dellaert, D. Fox, a D. H¨hnel, C. Rosenberg, C. N. Roby, J. Schutle, and D. Schultz, “Minerva: A Second Generation Mobile Tour-Guide Robot,” IEEE ACKNOWLEDGMENTS International Conference on Robotics and Automation (ICRA-99), June The authors would like to thank Mr. Lee Badger, the owner 1999, Antwerp, Belgium. [17] http://www.haptica.com/walker.html, Guido, Haptica of Lee’s MarketPlace, a grocery store in Logan, Utah, for Corporation. allowing them to use his store for testing RoboCart. The [18] http://www.opencyc.org, The OpenCyc Project: Formalized authors would like to thank Dr. Sarah Rule, Director of the Common Knowledge, Cycorp, Inc.