A set of new video analytics algorithms is described for automatic object detection and rule-based event recognition. The algorithms utilizes a 4D feature pyramid to model objects and the background in HD. A commercial version based TI's DaVinci DSP is embedded in intelligent IP-cameras and video encoders.

1. http://synesis.ru/ Embedded Video Analytics DSP Algorithms forDetection, Tracking and Recognition

2. HD Intelligent Network Video Media and Internet Face detection and recognition servers Intelligent Video Surveillance Intelligent cameras, encoders and DVRs Digital TV DVB receivers,STBs, PVRs,media centres

3. Efficient video surveillance (1) Accurateevent recognition correct classification false positives andfalse negatives response time documentation ?

4. Efficient video surveillance (2) Widespread infrastructure Cross-correlation of events captured by multiple cameras and other sensors Alert prioritization Distributed attacks(multiple point intrusions)

5. Efficient video surveillance (3) Operator productivity Keep attention focused Reduce subjectivism Increase response time

6. Efficient video surveillance (4) Cost of ownership Deployment Maintenance Telecom service charges Minimum team size Training Upgrade(Investment protection)

7. What is video analytics? X, Y, Z

8. Functions of video analytics Anti-tampering and operability monitoring Operational alerts Automatic priorities Automatic PTZ-camera targeting Event recording for instant forensic analysis Optimal usage ofnetwork bandwidth and storage memory

9. Solution: embedded video analytics Edge device transmits video andmetadata (object and its behaviour description) Zone 5intrusiondetected VIDEO EVENTDATABASE EVENT RULES METADATA

10. Embedded vs server analytics BOTTLENECK camera orencoder video management system or DVR compressedvideo & audio Embedded(front-end)analytics codecs video-analytics video management system or DVR ip-cameraor encoder Server(back-end)analytics metadata videoanalytics video and audio codecs

11. Video signal sources Network cameraAxis 211A Analoguestandard definition cameras(PAL/NTSC) Network cameras(standard and highdefinition) Thermal cameras Thermal cameraTitan-14

12. Wide angle perimeter surveillance(multiple tripwire alert levels)

13. Fence crossing detector

14. Apartment housing event recording

15. Directional detector

16. Running behaviour recognition

17. Time-based loitering behaviour recognition

18. Split target /abandon luggage detection

19. Group people tracking

20. Tampering and malfunction detectors Loss of signal Obstruction Out of focus and lens dusting Blackout and overexposure AE failure Lightingfailure

21. Upon a suspicious event… PTZ-targeting System notificationover IP network to VMS Sound and visual alarms, SMS etc ‘Dry contact’ signal High quality recording to local or remote storage (NAS) Analogue output to legacy systems (matrix or DVR)

22. Digital image stabiliser (antishaker) Eliminates video shakingcaused by wind and industrial vibrations Essential for analytics performance Differentiates the camera movementsfrom scene background/foreground movements

23. Video analytics components

24. Object tracker complexity complexity

25. Dynamic texture of the real world

26. Dynamic texture modelling OBJECT HAAR FEATURES BACKGROUND 4D-pyramid Featureprobability cloud α-channel (mask) for each object

27. People group tracking (Q4 2010) Feature cloud enablesobject tracking under partial visibility Z-buffer to identify object occlusions

28. Rule based behaviour recognitionEach zone is configured independently Zone entrance Zone exist Zone loitering:Staying overpredefined period of time Zone running: Exceeding a predefined speed Directional move within zone

29. Metadata sent over IP network / ONVIF Event type, data and time Zone or tripwire number 2D object feature: Position, size, area, speed Real 3D features Estimated from 2D featuresusing calibration data JPEGframe image withobject trajectory annotation

30. Videoanalytics calibration Two human figures define scale & angle Drag’n’drop calibration Tracking region 2D to 3D coordinate transform

31. Video analytics parameters Service detectors Antishaker Object tracker Contrast sensitivity Special sensitivity Min. stabilisation time Object filters Maximum object speed Min and max areas 1 2 3 4

32. Video analytics evaluation Methods and results

33. Video analytics public tests

34. Sterile Zone Performance 38 hours, PAL (720 x 576 x 25 fps), M-JPEG, 40 Mbps Number of true positive alarms: a = 432 False positivesalarms (typeI error): b =2 False negativesalarms (typeII error): с= 0

35. Resolution vs width field of view (FoV) 7-12 m 12-23 m 27-37 m

36. Maximum response time People walking and running 2 seconds People moving slowly(e.g. crawling) 10 seconds

37. Causes of false negatives(simple motion detectors) Unstable background decreasessensitivity of an adaptive detector DYNAMIC TEXTURE MODELING ALGORITHMSENABLE ROBUST OBJECT DETECTION IN A CHALENGING ENVIROMENT

38. Causes of false positives(basic motion detectors) Variable lighting Shadows from moving clouds and sun Moving trees, bushes and water Camera shaking Animals, birds and insects Object trajectory split and double detection Snow, rain, fog

39. Examples of false positives(simple motion detectors) BIRD RABBIT INSECT CAMERA SHAKING VIDEO ANALYTICS PREVENTS FALSE ALARMS CAUSED BY THESE FACTORS

40. Object trackingwhilst tree shadows moving

41. Performance estimation by3D security modeling 3D modeling building infrastructure control zones of camerasand third-party detectors treats (in space-time) Estimation of detection probabilities under variable external conditions day/night, fog, snow Video presentation ORIGINAL BUILDING 3D MODEL OF BUILDNG

42. Hardware reference designs Multifunctional video services and HD cameraswith embedded analytics

43. System-on-chip video analytics Videofilters Linux Video analytics HD H.264 codec 1080p

44. Dual channel video analytics encoder 4/11/2010 44 ANALOG + IPHYBRID TECHNOLOGY Two analogue inputs (BNC) Two managed outputs (BNC)and digital video over IP H.264 &MJPEG encoding Embedded video & audio analytics POE+and backup power ONVIF 1.01 support - 40⁰...+50⁰С Lightning guard

45. Dual channel video analytics encoder

46. Interfaces AUDIO OUT USB LAN I/O AUDIO IN RESET POWER BATTERY

47. HD video analytics camera

48. MJPEG vsH.264 compression DATAFLOW, MBPS RESOLUTION

49. Applications and use-cases Video analytics encoder

50. Integrated solution: Embedded video analytics Automatic PTZ targeting Unlimited, multizone sensor integration (I/O, RS485) Active illumination Two-way intercom Backup power &battery management Self-contained intelligence for perimeter security MB

51. Sophisticated landscape

52. Strategic infrastructure

53. Cost-effective upgrade oflegacy analogue infrastructure No cable or camera replacement required Increase storage efficiency by 10-100 times Automatic operational alerts Intelligent search using recorder events Future proof network surveillance via ONVIF

54. Local/backup storage Detachable video storage USB 2.5” hard drive or flash memory Accurate timestamp (NTP sync) Backup storage if NAS not available Portable player, video can be played on any PC

55. Unique selling position Fully embedded (DSP) implementation Real-time processing of uncompressed video HD/Megapixel resolution Highly scalable Unmatched performance in harsh environment dynamic texture engine Wide interoperability ONVIFcompliance

56. Example of customization Custom user interface Custom network and serial protocols Overlay text (POS, industrial etc) Custom DaVinci codecs(e.g. H.264 SVC) Custom video analytics

57. Future of video surveillance Multiple camera tracking using 3D model

58. Segmentation problemand object occlusions ‘Single camera’video analytics ‘Multiple camera’video analytics A B C A

59. i-LIDS multiple camera tracking scenario 2 3 4

60. 11/04/2010 www.synesis.ru 60 Video analytics + 3D modeling 3D model of a buildingand camera controlzones 1 2 Камера 2 Камера 1

61. OBJECT UNIQUE ID PRESERVED WHEN TRACKING FROM CAMERA TO CAMERA 11/04/2010 61

62. 3D trajectory reconstructed frommultiple video sources