Automotive Linux Summit / co-event LinuxConJapan 2013

1. “ Application Oriented Computer Vision
Pipelines for Automotive Industry”
Automotive Linux Summit '13, Tokyo
Kerem Caliskan
InfoDif
Oytun Eren Sengul
Tizen Turkey

2. About us
• 5 years old spin-off
• Pure R&D company
• 14 engineers, 6 administrative staff
• Graduates of reputable universities
• 6 MSc. and 3 PhD studies in the company
• Average experience: 6 years
2

3. I have a dream:

4. Facts 4
• Owning a vehicle is costly
Car prices
Fuel prices
Repair and maintenance costs
• Owning a car leads to security concerns about;
The family members
The vehicle as a property
Technology can provide more security and
comfort to vehicle owners

5. Questions 5
• Family members and/or co-workers sharing a vehicle ask:
 Does my daughter use her seat belt or just sit on it after
fastening it?
 Does my son drive securely?
 Who has halved the gasoline in our car?
 Can I record the violation of traffic rules with a camera on my
car?
 How can we prevent children riding on the front seat?
 Can my car automatically adjust the mirrors and the seats
besides setting a speed limit for pre-defined drivers?
 Who used the car and for how long?

6. Face off Application 6
1 2

7. Face off Application 7

8. Face off Application 8

9. Face Recognition –
Driver recognition
9
• We are able to develop a solution that can:
●
Keep track of driver identities and driving periods,
• Users of our solution will be able to:
●
Prevent undesired use of the car by children, etc.
●
Track the car’s fuel consumption,
●
Enjoy a more comfortable and secure drive.

10. Face Recognition –
Driver recognition
10
• Seat, mirror and speed limit adjustments for pre-defined drivers.
• When a car is used by more than 1 people it is possible to:
 Recognize the identity of the driver through a camera mounted behind
rear-view mirror,
 Automatically set his/her personal adjustments before driving starts.

11. Face Detection –
Driver drowsiness detection
11
• 30% of fatal crashes and 15% of serious injury crashes caused by driver
fatigue .
• Face tracking and detection capabilities: Virtual Makeover.
• Adaptation of algorithms used for Virtual Makeover to detect driver
drowsiness.

12. 12
• Driver state sensors to analyze driver’s sleep and concentration status
and point of concentration
●
Eye aperture
●
Eye position
●
Eye size
●
Face rotation
• Driver can be warned accordingly.
Face Detection –
Driver drowsiness detection

13. 13
Face Detection –
Driver drowsiness detection

14. Automated Seat Belt Check 14
• Inefficient SBR (seat belt reminder) mechanisms due to ignorance.
●
People tend to stop the alarm without fastening the belt:
●
Fastening seat belts to empty seats
●
Fake seat belt locks to stop audible SBRs.

15. Automated Seat Belt Check
with Image Processing
15
• Image processing-based solution for this: Automated Seat Belt Check.
• Images of driver and front-seat passengers will be captured by a camera mounted
behind rear-view mirror.
• We will be able to:
●
Detect specific edge points of seat belts on the image,
●
Determine if the belt is fastened or not,
●
Warn the driver if the belt is not fastened.

16. Automated Seat Belt Check
with Image Processing
16

17. Intruder Detection for Vehicles 17
• Security solution for object detection: Fence Intrusion Detection.
• Ready-to-use algorithms to detect unwanted:
●
object entrance
●
object exit
●
object dwell
within the border defined by the user and generate alarm.
• Minimizes false intrusion detections by user-defined point, time and
direction parameters.

18. Intruder Detection for Vehicles 18

19. Intruder Detection for Vehicles 19
• Cameras set up outside of the car do not consume much battery,
• They are operational even when the car is stopped.
• In case of an intrusion upon the user-specified border, the application will
generate an alarm and send via 3G:
●
The application will be able to inform the user for any potential threats
around his/her vehicle,
●
Recognizes the status of detected object as ‘stopped’, ‘dwelling’ and
‘moving’ as well.

20. Solution for Child Passenger Safety 20
• Although all children under 13 should ride in the back,
• Research has shown that 32% of all child loss is still among children
riding in front.
• The best way to minimize the risk is:
●
Making an age estimation for people sitting in the front seat before the
engine starts,
●
Preventing the vehicle to run if a baby/child passenger is detected in
the front seat.

21. Solution for Child Passenger Safety 21
• By using a camera mounted behind rear-view mirror, new technologies
capable of making age estimation.
• It made us experienced in face feature detection:
●
After detecting the eye and nose features of the face,
●
We analyze the pouches or wrinkle-like structures under the eye to
estimate the age.
• We can use our know-how for driver-safety purposes.

22. Assistance To Voluntary
Traffic Inspection
22
• It is getting popular among civils to voluntarily inspect and report traffic
infringements.
• By adapting our existing motion detection algorithms (object entrance, exit, dwell,
stop, object in speed limit) to the field,
• The proposed solution will be able to detect and alert the driver in case the front
car:
●
Cuts in,
●
Weaves through traffic,
●
Overtakes on right direction.

23. Assistance To Voluntary
Traffic Inspection
23

24. Assistance To Voluntary Traffic
Inspection
24
• Traffic inspection assistance solution to be capable of:
●
Helping voluntary traffic inspection by warning the driver,
●
Notifying the driver to take necessary action against any possible
threat,
●
Supporting safe flow of traffic by minimizing accidents
●
Leading people to drive more carefully

25. Suspicious / Unattended
Package Detection
25
• Combining motion detection, object tracking and face recognition abilities,
●
We developed i-bex Video Analytics Platform
●
i-bex is able to:
●
Detect suspicious/unattended objects in crowded places and,
●
Match the object with the person who left it.

26. Suspicious / Unattended
Package Detection
26

27. Fog Elimination 27
• Fog elimination filters of Capra Image Processing Platform:
●
Eliminates fog from the image,
●
Increases visibility range by 30-40%

28. Fog Elimination 28

29. Speed Estimation 29
• Using our speed estimation capability, we are able to:
●
Detect the car on the image,
●
Measure its speed,
●
Warns you if it coming up like a crazy,
●
No need to worry, if he escapes after crash, because his plate already
token and waiting your confirmation to informs directly to police.

30. Speed Estimation 30

31. Lane Detection 31
• Existing lane warning / keeping systems alert the driver or takes
necessary actions (corrective steering or braking) when:
●
Vehicle enters to blind spot while switching lanes,
●
Vehicle leaves its lane,
●
The driver changes the lane without turning on the signal, etc.
• However, existing systems may not always work very efficiently because:

32. Lane Detection 32
●
Warnings may distract driver’s attention and cause panic,
●
In case of stopping the driving for a while, lane keeping system may
become deactivated.
• Inefficiency of these systems may cost a life.
• We have digged up all lane-related concerns of drivers
• Combine them into a custom Lane Detection solution that meets the
needs.

33. Lane Detection 33
• for example;
the OCR algorithms used for one of world's biggest operator,
• Easily adaptable to lane detection,
• More effective for lane-detection compared to standard algorithms.

34. Lane Detection 34

35. Platforms 35
• Operating systems are not well-configured for image processing, so:
• To be able to develop all these solutions in an effective way, we:
●
Worked hard to develop our own real time image processing
framework called ‘CAPRA’,
●
Use multi-threaded structure of image processing pipelines inside this
framework,
●
Provide thread-level synchronization in our solutions
●
We have developed cross-platform software so far, it can also run on
Linux and Windows. But of course we've get better results on Linux :)

36. Platforms 36
• Using CogniVue new generation embedded vision processing hardware
because they:
●
Meet the needs of digital signal processing problems,
●
Accelerate 3D graphics processing,
●
Specialized for signal processing in embedded systems,
●
Enable performance gain

37. 37
Products
• Generic software frameworks
• Provide image processing ability to users (software developers,
integrators etc.)
• Low cost of ownership and maintenance
• x86 and ARM based solutions, testing on Tizen IVI

38. Q/A
Kerem Caliskan
kcaliskan@infodif.com
Thank you...
Oytun Eren Sengul
oytuneren@linux.com