Current state of art contains several methods to achieve intelligent tracking. Some methods are machine learning oriented. In these methods, activities are learnt from the context in an unsupervised or semi supervised manner. One other method is description based event recognition. In the heart of the method , describing scenarios wrt activities employed. For the description, a language is necessarily needed. There are mathematical languages in which logic is used to represent activities and their relations.Also some graphical languages such as hidden markov models, state machines, state charts are being used. Some textual languages proposed as well.

1. VIDEO ACTIVITY RECOGNITION VIA
RGB-D
Francois BREMOND
Carlos fernando Crispim junior
Nazli TEMUR
1

2. 1) General Project Description
2) Contribution
3) Conclusion
4) Proposal
5) Bibliography
CONTENT
& Project
& Motivations
& Challenges
& Goals
2

3. PROJECT
General Project Description
3

4. PROJECT
➤ Domain of the project is real-time, automatic video
sequences understanding.
➤ In this work, RGBD sensor is used to acquire 3D images,
detect people and recognize interesting activities.
➤ The SUP library, which is developed by STARS Team is
used for detection, tracking and recognition of the people.
➤ For privacy purposes we have focused ONLY on the depth
information of RGBD cameras
4

5. INPUT / DATASET
➤ Dataset : Home Care [Nursing Home] recording on depth data, videoclips.
➤ 1 day[24 hours] of video is chosen out of months of recording of unconstraint
recording.
➤ Already annotated video is used after verification.
➤ Total Annotated Events 197 with the numbers of
Enter_Restroom: 18
Exit_Restroom : 19
Leave_Area_In_Bedroom : 75
Enter_Area_In_Bedroom : 76
Sitting : 9
5

6. MOTIVATION
General Project Description
6

7. MOTIVATION
➤ Population aging is a motivation
to develop intelligent
technologies to support the life
of older people , especially via
monitoring systems.
➤ Activity recognition is a key way
for better assessment of
monitoring.
➤ So that my motivation is to
improve activity recognition for
providing comfort to the elderly
people’s daily life.
http://www-sop.inria.fr/members/Carlos-Fernando.Crispim_Junior/demos.html 7

8. CHALLENGES
General Project Description
8

9. CHALLENGES
➤ People Detection
• Occlusions[Perspective challenge]
12
3
9

10. CHALLENGES
➤ People Detection
• Noises
1
2
10

11. CHALLENGES
➤ People Tracking
• Re-identification
• Overcome Occlusions
• Overcome Noise
1
2
3
11

12. CHALLENGES
➤ Event Recognition
• How to model activities
• How to recognition human
activities in real-time settings, past-
present approach for video analysis
12
3
12

13. CHALLENGES
➤ Event Recognition
• How to model activities
• How to recognition human activities
in real-time settings, past-present
approach for video analysis
34
5
13

14. CHALLENGES
➤ Event Recognition
• How to estimate target destination
• How to maintain different actors’ speed
to not be in trouble with duration
• How to infer actors interaction with
ambiguous, non-observed parts of the
scene
A
B C
14

15. GOAL
Evaluate Current Event Recognition Pipeline for Activity Recognition in
Unconstrained Environment
Study the event modeling language to implement new models.
Study the limitations of existing solutions.
Propose more suitable event models.
TO DO
15

16. CONTRIBUTIONS
16

17. CONTRIBUTIONS
➤ Proposition of a new approach to reason about actors that
leave/enter the scene in the midst of noisy people
observations.
➤ - Previously: Empty scene (hard constraint)
➤ - Proposition: Increase/Decrease in the number of actors
Note: Empty Scene restricts the events to a single actor.
Thanks to newly proposed scenarios the event recognition can
be done when multiple actor/noise present on the scene.
17

18. CONTRIBUTIONS
➤ Proposition of a new layout for the spatial zones.
18

19. EXAMPLE SCENARIOS OF THINTH LANGUAGE
CompositeEvent(EXIT_RESTROOM,
PhysicalObjects((p1 : Person), (z1 : Zone), (sc : Scene))
Components((c1: CompositeEvent Disappear(p1,sc))
(c2: PrimitiveEvent IN_AREA_IN_RESTROOM(p1,z1)))
Constraints((c1 justBefore c2)
(duration(c2) > 0.5))
Alarm ((Level : URGENT))
)
CompositeEvent(Disappear,
PhysicalObjects((p1 : Person), (sc : Scene))
Components((c1: PrimitiveState Person_exists(p1))
(c2: PrimitiveState Negative_Change_Actor_Number(sc)))
Constraints(
(c1 justBefore c2)
(!(Exist(p1))))
Alarm ((Level : URGENT))
)
PrimitiveState(Negative_Change_Actor_Number,
PhysicalObjects((sc : Scene))
Constraints ((AttributeChange(sc->NumberOfActors) = 2))
Alarm ((Level : URGENT))
)
ATTRIBUTES
➤ Composite Event
➤ Primitive Event
➤ Primitive State
19

20. CONCLUSION
20

21. CONCLUSION
In this project, description based video activity recognition and evaluation is
studied.
Improvement proposals are suggested on Event Modeling and Trajectory
Analysis Topics.
Thanks to the newly proposed event models, it is possible to recognize events in a
scene where multiple actors and noises are present.
For the evaluation process, we benefited from several tool such as SUP Event
Recognition Platform, Thinth Event Modeling Language ,Viseval ,Viper,
KreateTool.
While evaluating , we have used True Positive,False Positive and False Negative
measurements; F1 score, precision, recall metrics.[TP,FP,FN]
Evaluation on previously defined event models are completed.
Currently, latest assessment on newly defined zones and event models is ongoing
because of Viseval tool related error.
21

22. PROPOSALS
22

23. PROPOSALS
➤ Use of Trajectory is investigated. Trajectory usage is
promising, but to be able to implement a precise solution,
state of art for the trajectory need to be reviewed wrt
occlusion and noise reduction.
23

24. BIBLIOGRAPHY
24

25. [1] C. Crispim-Junior, K. Avgerinakis, V. Buso, G. Meditskos, A. Briassouli, J. Benois-Pineau, Y. Kompatsiaris and F. Bremond.
Semantic Event Fusion of Different Visual Modality Concepts for Activity Recognition, Transactions on Pattern Analysis and
Machine Intelligence - PAMI to appear, 2016.
[2] C. Crispim-Junior, V. Bathrinarayanan, B. Fosty, A. Konig, R. Romdhane, M. Thonnat and F. Bremond. Evaluation of a
Monitoring System for Event Recognition of Older People. In the 10th IEEE International Conference on Advanced Video and
Signal-Based Surveillance 2013, AVSS 2013, Krakow, Poland on August 27-30, 2013.
[3] C. Crispim-Junior, B. Fosty, R. Romdhane, F. Bremond and M. Thonnat. Combining Multiple Sensors for Event Recognition
of Older People. In the 1st ACM International Workshop on Multimedia Indexing and Information Retrieval for Healthcare,
MIIRH 2013, Copyright 2013 ACM 978-1-4503-2398-7/13/10, http://dx.doi.org/10.1145/2505323.2505329, Barcelona, October
22, 2013.
[4] Alberto Avanzi, Francois Bremond, Christophe Tornieri and Monique Thonnat, Design and Assessment of an Intelligent
Activity Monitoring Platform, in EURASIP Journal on Applied Signal Processing, special issue in "Advances in Intelligent Vision
Systems: Methods and Applications", 2005.
[5] E. Corvee and F. Bremond. Haar like and LBP based features for face, head and people detection in video sequences. In the
International Workshop on Behaviour Analysis, Behave 2011, Sophia Antipolis, France on the 23rd of September 2011.
[6] C. Crispim-Junior and F. Bremond. Uncertainty Modeling Framework for Constraint-based Elementary Scenario Detection in
Vision System. In the First International Workshop on Computer vision + ONTology Applied Cross-disciplinary Technologies in
conjunction with ECCV 2014, CONTACT-2014, Zurich, Switzerland, September 7th, 2014.
[7] A. König, C. Crispim, A. Covella, F. Bremond, A. Derreumaux, G. Bensadoum, R. David, F. Verhey, P. Aalten and P.H. Robert.
Ecological Assessment of Autonomy in Instrumental Activities of Daily Living in Dementia Patients by the means of an
Automatic Video Monitoring System, Frontiers in Aging Neuroscience - open access publication - http://dx.doi.org/10.3389/fnagi.
2015.00098, 02 June 2015
25

26. THANKS!
26

27. SHOWCASE
27