Kotani, K., Sun, W., Kitani, T., Shibata, N., Yasumoto, K., Ito, M.:Inter-Vehicle Communication Protocol for Cooperatively Capturing and Sharing Intersection Video, Proc. of 2nd IEEE Intelligent Vehicular Communications System Workshop (IVCS'10), (CD-ROM), Jan. 9th, 2010. DOI:10.1109/CCNC.2010.5421635 (Jan. 2010). http://www.aist-nara.ac.jp/~sunweihua/papers-fullversion/I-10-01-02.pdf For accident prevention at intersections, it is useful for drivers to grasp the position of vehicles in blind spots. This can be achieved without infrastructure if some vehicles passing near the intersection capture and share live video of the intersection through inter-vehicle communications. However, such video streaming requires a congestion control mechanism. In this paper, aiming to let a driver grasp the situation at an intersection, we propose a method to select vehicles that send a video in order to generate a live bird’s-eye-view video of the intersection. In our method, each vehicle at an intersection exchanges information with others, such as the sub-areas of the intersection it captures, the quality of its video, and its position and speed. Based on the exchanged information, each vehicle autonomously judges whether it should send its video or not. Through simulation with a QualNet simulator, we confirm that our method achieves a good video arrival rate and video quality sufficient for practical use.

1. Kazuya Kotani†1, Weihua Sun†1, Tomoya Kitani†2,
Naoki Shibata†3, Keiichi Yasumoto†1, Minoru Ito†1
†1 Nara Institute of Science and Technology (NAIST), Japan
†2 Shizuoka University, Japan
†3 Shiga University, Japan
1
Inter-Vehicle Communication Protocol for
Cooperatively Capturing and
Sharing Intersection Video
(revised slides)

2. 2
Goal: Allow drivers to grasp vehicle positions
in blind spots at intersections
Overview
Proposed Method
• Vehicles shoot videos from onboard cameras
• Each vehicle decides if it should send its video wirelessly
• according to available bandwidth, etc.
• Each vehicle receives videos from multiple vehicles and
compose them into a Bird's Eye View Video (BEVV)
• In order to reduce latency, It does not use multi-hop
communication

3. 1. Background
2. Related Work
3. Proposed Method
4. Experimental Results
5. Conclusion
3
Outline

4. 4
We propose a system that allows drivers to see
vehicle positions in blind spots in an intuitive way
Intersection
43.9％
Local Street
26.8％
Vehicles in blind spots
[1] ITS: “Ministry of Land, Infrastructure, Transport and Tourism,”
http://www.mlit.go.jp/road/ITS/.
Background
City area
74.8％
 43.9% of traffic accidents happen at
◦ intersections in city area[1]
 They are collisions between
◦ vehicle and vehicle
◦ vehicle and pedestrian
 Most of these are caused by

5. 1. Background
2. Related work
3. Proposed method
4. Experimental results
5. Conclusion
5
Outline

6.  DSSS(Driving Safety Support Systems) project [2]
◦ Gathers other vehicles’ position with inter-vehicle comm.
◦ Vehicles without communication devices are not displayed
◦ Positions are not accurate because of GPS positioning error
6
[3] Ota et al. : ``Visual Reconstruction of an Intersection by Integrating Cameras on Multiple Vehicles,’’ Proc. on
Machine Vision Applications (MVA2007), pp.335–338(2007).
[2] Honda Motor Co., Ltd.: “Honda Technology,” http://www.honda.co.jp/news/2009/4090219.html?from=rss.
 Method to compose a
birds-eye-view video(BEVV)[3]
◦ Multiple videos are taken from multiple cars from different angles
◦ Videos can be composed into one BEVV
◦ The method includes no mechanism for exchanging videos
Related Work

7. 7
IEEE802.11b, 200 Kbps
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10
PacketArrivedRatio
[%]
Number of Vehicles
Problems with this method
• Vehicle density can be too high
• Wireless bandwidth is not insufficient
We need to carefully select vehicles that send videos
to save wireless bandwidth
Problem with a straightforward approach for
exchanging captured video
1. Vehicles broadcast request messages before turning right at intersection
2. All vehicles in intersection capture and send videos wirelessly
3. Vehicles receive videos and compose BEVV
The straightforward method

8.  Objectvie
◦ Drivers need high quality view of the blind spots
 Approach
◦ Assign priority to each vehicle for sending video
◦ The following vehicles are assigned high priorities
 Vehicles that can capture the requested areas
 Vehicles that can capture high quality videos
◦ Vehicles send out the captured video
according to their priorities
8
Key Idea for Selecting Vehicles
0
7
56
0
4
8
6
5

9. 1. Background
2. Related work
3. Proposed method
4. Experimental results
5. Conclusion
9
Outline

10.  We assume that it is possible to compose a BEVV from
videos taken from any different angles
 We conducted preliminary experiment with scaled down
model of intersection
convert
10
compose
Assumptions
Captured
on camera
Converted
video
frame
Multiple video
frames
Bird’s eye view
frame

11.  An intersection is divided into rectangular cells
◦ Each cell is defined by its geographical position
◦ Each vehicle knows which cell it is located
◦ Each vehicle know cells that can be captured by its camera
 Finer position can be determined by visually analyzing
captured video[4]
11
Requesting
vehicle
[4] Alberto et al. : `` Multi-Resolution Vehicle Detection using Artificial Vision,’’
Proc. on IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV’04), pp.310–314(2004).
Dividing Intersection into Cells
Request cells

12.  Vehicles have the following equipment
◦ On-board camera
◦ Car navigation system with GPS
◦ Wireless LAN communication device
◦ On-board computer
 On-board computer can sense the state of
indicator (going straight, turning left or right)
12
Assumptions on Vehicle

13. 13
1. All vehicles periodically
exchange Hello
Message
2. Requesting vehicle
broadcasts
Request Message
3. Each vehicle which received
Request Message calculates all
vehicles’ Priority
86
7
6
0
00
70
0
7
4. Each vehicle decides if it
should send video frames
Requesting vehicle
How Proposed Method Works

14. There are two phases in the proposed method
 Information Exchange Phase
◦ Vehicles exchange messages to get the information from the
surrounding vehicles
 Vehicle Selection Phase
◦ Videos are sent utilizing available bandwidth as much as
possible
◦ Priority is calculated in this phase
14
Two phases in the proposed method

15. Goal : Each vehicle knows other vehicles’ information
Vehicles close to an intersection exchange the following
messages
 Hello Message
◦ All vehicles inform their own status to other vehicles
 position, speed, capturing cells, video quality,
number of video frames already sent
 Request Message
◦ Requesting vehicle sends requested cells to other vehicles
 items of Hello Message, requested cells
◦ When a vehicle receives a Request Message, it transits to Vehicle
Selection Phase
15
Information Exchange Phase

16. Goal : Making vehicles send videos utilizing available
bandwidth as much as possible
 Vehicles need to know the usage of communication
bandwidth
◦ The usage of communication bandwidth is defined by the ratio
of
 To decide if each vehicle should send videos
◦ Calculate the priority based on the following items
 Requested cells that can be captured by the camera on vehicle
 Video quality for each cell
◦ Priority is the weighted sum of these values
Preliminary experiment
 We conducted preliminary experiment to determine these weights 16
Vehicle Selection Phase
Number of actually received video frames
Number of video frames informed in Hello messages

17. Deciding if each vehicle sends video
 Each vehicle calculates priorities of vehicles
independently
◦ Usually, the results of calculations are almost same
 For each direction, the vehicle with the highest
priority decides to send the video
◦ For each direction, usually the car closest to the
center of intersection starts sending video
◦ Then, the cars following the first car sends their
videos in turn
◦ The priority is dynamically changed according to the
positions of moving vehicle
17
Vehicle Selection Phase(contd.)

18. 1. Background
2. Related work
3. Proposed method
4. Experimental results
5. Conclusion
18
Outline

19.  Goal
◦ To see if the proposed method can select vehicles that capture
requested cells in high quality
Evaluation criteria
1. The number of video frames covering each cell in requested cells
2. The number of video frames covering requested cells from each
driving direction
3. The priority of video frame received by request vehicle
19
Experimental Evaluation

20.  We used QualNet[5]
◦ Created terrain data to imitate actual intersection in Kyoto, Japan
◦ Requesting vehicle goes into intersection ⇒ turns to the right
◦ The number of requesting vehicle is one
Transmitting bit rate: 200 [Kbps]
20
Settings

21.  Simulation parameters
 Packet parameters
21
item value
The number of vehicles 60
Requested cells 3 cells
Wireless LAN standard IEEE802.11b
Vehicle density Dense, Middle, Sparse
Carrying rate of equipments 100%, 60%, 30%
Packet name Packet size
Transmission
interval
Hello Message 300[byte] 0.5[s]
Request Message 300[byte] 0.5[s]
Video frame 1666[byte] 0.067[s]
Parameter setting of experiment

22.  We compared the following methods for
selecting vehicles to send their videos
i. Proposed method
ii. All vehicles
All vehicles near intersection send video
iii. Vehicles closest to the center of intersection
The vehicle closest to the center of intersection from each
driving direction sends video
Measured evaluation criteria with these three methods
22
Comparison of Three Methods

23. 23
Request
vehicle
Request
cells
green1
green2
red
2
red
1
1 2
3
Evaluation experiment
Driving directions and request cells

24.  In order to see vehicles in blind spot by video, video
quality of at least 10[fps] in average is necessary
 RNF is the minimum number of received frames that
satisfies the above condition for each cell
◦ Simulation time and RNF value in each density setting are
below
24
Required Number of Frames (RNF)
Vehicle density Dense Middle Sparse
Simulation time 35[s] 58[s] 88[s]
RNF(each direction) 350[frame] 580[frame] 880[frame]
RNF(each cell) 1400[frame] 2400[frame] 3520[frame]
×4 (driving directions)

25.  In Dense setting with 100% carrying rate
◦ Proposed method achieves RNF on all request cells
◦ All vehicles doesn’t achieves RNF on any cells
◦ Vehicles near the center achieves RNF on two cells
25
Number of video frames (Request cells) 1/3
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrames
Request cells
Proposed method
All vehicles
Vehicles near the center
RNF

26.  In Dense Middle and Sparse setting with 100% carrying
rate
◦ Proposed method sends more frames than any other method
regardless of vehicle density
26
Number of video frames (Request cells) 2/3
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrames
Request cells
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
cell1 cell2 cell3
NumberofReceivedFrame
Request cells
0
1000
2000
3000
4000
5000
6000
7000
8000
cell1 cell2 cell3
NumberofReceivedFrame
Request cells
Proposed method
All vehicles
Vehicles near the
center
RNF
Dense Middle Sparse

27.  In Dense setting, with 100, 60, 30% carrying ratio
◦ Proposed method provides more frames than other methods
◦ Request cells achieving RNF decreases as carrying rate decreases
27
Number of video frames (Request cells) 3/3
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrames
Request cells
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrame
Request cells
0
500
1000
1500
2000
2500
3000
call1 cell2 cell3
NumberofReceivedFrame
Request cells
Proposed
method
All vehicles
Vehicles near
the center
RNF
100% 60% 30%

28.  In Dense setting with 100% carrying rate
◦ Proposed method achieves RNF from all directions
◦ Other methods achieve RNF from one or two directions
28
Number of video frames (Driving direction)
0
200
400
600
800
1000
1200
1400
1600
1800
green1 green2 red1 red2
NumberofRecievedFrame
Driving Direction
Proposed method
All vehicles
Vehicles near the center
RNF

29.  In Dense setting with 100% carrying rate
◦ There are more high priority frames by proposed method than other
methods
29
0
10
20
30
40
50
60
70
80
90
100
24~30 18~24 12~18 6~12 0~6
CummulativeDensityof
Frmae[%]
Priority
Proposed method 100%
All vehicles 100%
Vehicles near the center 100%
 Effective in Dense setting with 60% or more carrying rate
 More efficient than other methods in all environments
Conclusions of experiment results about proposed method
Priority
Cumulative distribution function (CDF) graph
Proposed method can make BEVV which contains request cells in high qua

30.  Vehicle selection method
◦ For creating a bird's eye view video by letting selected
vehicles efficiently capture and exchange video
◦ It assigns priorities to send videos to other vehicles
◦ Vehicle with higher priority sends videos within the
available communication bandwidth
 Results
◦ Proposed method is especially effective in the dense
setting with 60% or more arrival rate
30
Conclusion

31. Kotani, K., Sun, W., Kitani, T., Shibata, N.,
Yasumoto, K., Ito, M.:Inter-Vehicle Communication
Protocol for Cooperatively Capturing and Sharing
Intersection Video, Proc. of 2nd IEEE Intelligent
Vehicular Communications System Workshop
(IVCS'10), (CD-ROM), Jan. 9th, 2010.
DOI:10.1109/CCNC.2010.5421635 [ PDF ]
31