Presentation given at ICPRAM 2013, International Conference on Pattern Recognition, Barcelona, Spain, February 2013
ABSTRACT: We propose a novel gesture spotting approach that offers a comprehensible representation of automatically inferred spatiotemporal constraints. These constraints can be defined between a number of characteristic control points which are automatically inferred from a single gesture sample. In contrast to existing solutions which are limited in time, our gesture spotting approach offers automated reasoning over a complete motion trajectory. Last but not least, we offer gesture developers full control over the gesture spotting task and enable them to refine the spotting process without major programming efforts.
Paper:http://www.academia.edu/2157798/Declarative_Gesture_Spotting_Using_Inferred_and_Refined_Control_Points
Asymmetry in the atmosphere of the ultra-hot Jupiter WASP-76 b
Declarative Gesture Spotting Using Inferred and Refined Control Points
1. Declarative Gesture Spotting Using
Inferred and Refined Control Points
Department of Computer Science, Vrije Universiteit Brussel (VUB)
Lode Hoste - lhoste@vub.ac.be
Brecht De Rooms - bderooms@vub.ac.be
Beat Signer - bsigner@vub.ac.be
1
16. Automatically Infer Control Points
One representative sample
- start with the first point
- search for the smallest angle
- within a given time-frame
Z gesture
16
17. Automatically Infer Control Points
One representative sample
- start with the first point
- search for the smallest angle
- within a given time-frame
Z gesture
17
18. Automatically Infer Control Points
One representative sample
- start with the first point
- search for the smallest angle
- within a given time-frame
- refine control points
(optionally)
Z gesture
18
20. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D) there is a start point
?p1
20
21. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D)
3 ?p2 (Point2D) there is a second point ?p1 ?p2
21
22. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D)
3 ?p2 (Point2D) ?p1 ?p2
4 (test (< ?p1.time ?p2.time))
temporal constraint:
that comes after the start point
22
23. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D)
3 ?p2 (Point2D) ?p1 ?p2
4 (test (< ?p1.time ?p2.time))
5 (test (inside_control_point ?p1 ?p2 185 4 76))
spatial constraint:
23
24. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D)
3 ?p2 (Point2D) ?p1 ?p2
4 (test (< ?p1.time ?p2.time))
5 (test (inside_control_point ?p1 ?p2 185 4 76))
4
185
spatial constraint:
.. a circular area with centerpoint
x: 185 y:4 (pixels)
24
25. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D)
3 ?p2 (Point2D) ?p1 ?p2
4 (test (< ?p1.time ?p2.time))
5 (test (inside_control_point ?p1 ?p2 185 4 76))
4
185
spatial constraint:
.. a circular area with centerpoint
x: 185 y:4 (pixels)
and radious 76
25
26. Comprehensible Code Generation
Based on the inferred control points:
1 (defrule symbol_z
2 ?p1 (Point2D)
3 ?p2 (Point2D) ?p1 ?p2
4 (test (< ?p1.time ?p2.time))
5 (test (inside_control_point ?p1 ?p2 185 4 76))
4
185
26
31. A State Machine is Not Enough
template continuous stream
State 1
State 1 State 2
State 3 State 4
31
32. A State Machine is Not Enough
template continuous stream
State 2
State 1 State 2
State 3 State 4
32
33. A State Machine is Not Enough
template continuous stream
State 2
State 1 State 2
reset state machine?
State 3 State 4
33
34. A State Machine is Not Enough
template continuous stream
State 1
State 1 State 2
reset state machine?
State 3 State 4
34
35. A State Machine is Not Enough
template continuous stream
State 1
State 1 State 2
State 3 State 4
35
36. A State Machine is Not Enough
template continuous stream
State 1
State 1 State 2
State 3 State 4
36
37. A State Machine is Not Enough
template continuous stream
State 1 State 2
gesture overshot
missed gesture
State 3 State 4
37
38. A State Machine is Not Enough
template continuous stream
State 1 State 2
State 3 State 4
38
39. A State Machine is Not Enough
template continuous stream
State 1 State 2
refused by gesture
classification
State 3 State 4
39
40. A State Machine is Not Enough
template continuous stream
State 1 State 2
refused by gesture
classification
State 3 State 4
overlapping gesture
missed gesture
40
77. Extending the ‘Flick Right’ Definition
?p1 ?p2 ?p3
(not
(and
(Point2D (y ?b y) (time ?b time)) … there should not be a point where the …
(test (< ?b time ?p3.time))
(test (> ?b time ?p1.time))
(test (> (abs (- ?p1.y ?b y)) 245))))
77
78. Extending the ‘Flick Right’ Definition
?p1 ?p2 ?p3
(not
(and
(Point2D (y ?b y) (time ?b time)) … there should not be a point where the …
(test (< ?b time ?p3.time))
… time lies in between ?p1 and ?p3 …
(test (> ?b time ?p1.time))
(test (> (abs (- ?p1.y ?b y)) 245))))
78
79. Extending the ‘Flick Right’ Definition
y
?p1 ?p2 ?p3
y
(not
(and
(Point2D (y ?b y) (time ?b time)) … there should not be a point where the …
(test (< ?b time ?p3.time))
… time lies in between ?p1 and ?p3 …
(test (> ?b time ?p1.time))
(test (> (abs (- ?p1.y ?b y)) 245)))) … for which y is greater than 245.
79
81. Future Work
Scale / rotational Visual 3D gesture
invariant spotting definitions
81
82. Summary
lhoste@vub.ac.be bderooms@vub.ac.be bsigner@vub.ac.be
Auto infer control points Generate Spotting Code Call classification
• Extensive spotting: support for real-time continuous streams.
• No gesture overshooting: support for overlapping submatches.
• Comprehensible: easy to understand and to correct.
• Extensible: allows for expert adjustments.
82
83. Lode Hoste, Brecht De Rooms
and Beat Signer, Declarative
Gesture Spotting Using
Inferred and Refined Control
Points, Proceedings of
ICPRAM 2013, International
Conference on Pattern
Recognition, Barcelona, Spain,
February 2013
83