Sift based arabic sign language recognition aecia 2014 –november17-19, addis ababa, ethiopia- tarek-gaber

SIFT-BASED ARABIC SIGN LANGUAGE
RECOGNITION (ArSL) SYSTEM
By
Alaa Tharwat1,3
And
Tarek Gaber2,3
1Faculty of Eng. Suez Canal University, Ismailia, Egypt
2Faculty of Computers & Informatics , Suez Canal University, Ismailia, Egypt
3Scientic Research Group in Egypt (SRGE), http://www.egyptscience.netSuez Canal University
AECIA 2014 –November17-19, Addis Ababa, Ethiopia

Introduction
Proposed Method
Agenda
General framework
Feature extraction
Classification
Experimental Results
Conclusions
3

Introduction
Proposed Method
Agenda
General framework
Feature extraction
Classification
Conclusions
4

Introduction: Why ArSL
• Help vocally disabled people
to speak freely.
• Easy way of communication
with non-mute people.
• ArSL is the natural language
for deaf like spoken language
to vocal

Introduction: Aim of the work
 Design a sign language recognition approach to transcribe
sign gestures into meaningful text or speech so that
communication between deaf and hearing society can
easily be made.
الشارع السيارة

What is ArSL?
 Translating ArSL to spoken language, i.e. translate
hand gestures to Arabic characters
 Sign Language hand formations:
 Hand shape
 Hand location
 Hand movement
 Hand orientation

Introduction: Types of ArSL
1- Vision-based Approach
Requires special set up for camera, but needs some
preprocessing and computational to extract features.
Extract Features
Collect gestures
Classification
Decision

Introduction: Types of ARSL (Continue)
2-ElectronicGlove-based Approach
Inconvenience of gloves, but ease of signal extractions
The Electronic-gloves consists of:
• 22 sensors
• Light weight
• Flexible

Introduction
Proposed Method
Agenda
General framework
Feature extraction
Classification
Conclusions
10

Proposed Method: General Framework
Training Images Testing Images
SIFT Feature Extraction Method
Feature Vectors
Difference
of
Gaussian
Pyramid
KeyPoints
detection
Unreliable
KeyPoints
Eliminatio
n
Orientatio
n
Assignme
nt
Descriptor
Computatio
n
Feature Vectors
Matching LDA

Proposed Method: General Framework
 Training phase
 Collecting all training images (i.e.
gestures of Arabic Sign Language).
 Extracting the features using SIFT
 Representing each image by one feature
vector.
 Applying a dimensionality reduction
(e.g, LDA) to reduce the number
features in the vector
 Testing phase
 Collecting the testing image,
 Extract the features
 Feature vector is projected on LDA
space.
 Applying machine learning techniques
for classifying the test feature vector to
decide whether the animal is identified
or not).

Proposed Method: Feature Extraction
Feature Extraction SIFT (Scale Invariant Feature Transform
SIFT feature extraction algorithm
consists of the following steps:
• Creating the Difference of Gaussian Pyramid
(Scale-Space Peak Selection)
• Extrema Detection
• Unreliable Keypoints Elimination
• Orientation Assignment
• Descriptor Computation Keypoints or Extrema
extracted from one image
(gesture) using SIFT
algorithm

Matching between two getures based on SIFT features

The Number of features extracted by SIFT depends its
parameters which has been considered in our experiment:
• Peak Threshold (PeakThr)
• patch size (Psize)
• number of angels (Nangels) and number of bins (Nbins)

Proposed Method: Classification Techniques
 We have used the following classifiers assess their performance
with our approach :
 SVM is one of the classifers which deals with a problem of high dimensional
datasets and gives very good results.
 K-NN: unknown patterns are distinguished based on the similarity to known
samples
 Nearest Neighbor: Its idea is extremely simple as it does not require learning

Introduction
Proposed Method
Agenda
General framework
Feature extraction
Classification
Conclusions
17

Experimental Results: Dataset
 We have used 210 gray level images
with size 200x200.
 These images represent 30 Arabic
characters, 7 images for each
character).
 The images are collected in different
illumination, rotation, quality levels,
and image partiality.
A sample of collected ArSL gestures
representing different characters .

Experimental Scenarios
We have designed three experiment Scenarios:
 To select the most suitable parameters.
 To understand the effect of changing the
number of training images.
 To prove that our proposed method is
robust against rotation
 To prove that our proposed method is
robust against occlusion.

Experimental Results – 1st Scenario: Selecting SIFT parameters
PeakThr Psize Nangels
Accuracy results (in %) of our approach based on different SIFT
parameters
Classifier
s
32x3 2 4 8
2
16x1
6
0 0.1 0.2 4x4 8x8
NN 100 97.7 94.2 94.2 99.2 100 93.2 94.2 98.9 100
K-NN 100 98.9 96.3 96.3 99.2 100 93.6 96.3 98.9 100
SVM 100 99.2 98.9 97.7 100 100 94.2 96.3 98.9 100

Experimental Results – 2nd Scenario: Different Training No. of images
Classier No. of Training Images
5 3 1
Min. Dist. 100 99.2 98.9
k-NN (k=5) 100 98.9 98.9
SVM 100 99. 98.9
Accuracy results (in %) of our approach using different training
images

Experimental Results – 3rd Scenario: Rotated images
F.E.M. Matching Angles of rotation (o)
0 45 90 135 180 225 270 315
Min Dist. 100 98.9 97.8 96.7 100 97.8 100 98.9
SIFT
k-NN_5 100 100 100 96.7 100 98.9 100 100
SVM 100 100 98.9 98.9 100 98.9 100 100
Accuracy in (%) of our approach when rotated images are used

Experimental Results – 4th Scenario: Occluded images
F.E.M. Matching Percentage of Occlusion
Horizontal Vertical
20 40 60 20 40 60
SIFT Nearest Neighbor 98.9 93.3 34.4 98.9 95.6 32.2
k-NN_5 97.8 95.6 38.9 97.8 96.7 53.3
SVM 98.9 95.6 52.2 98.9 96.7 45.6
Accuracy of cattle identification based on image occlusion
(%)

A comparison between proposed system and previous systems.
Author Accuracy in (%)
K. Assaleh et al. [1] 93.5
Al-Jarrah et al. [6] 94.4
Al-Jarrah et al. [9] 97.5
Mohandes et al. [12] 87
Our proposed 99
Accuracy results (in %) of our approach using different training
images

Introduction
Proposed Method
Agenda
General framework
Feature extraction
Classification
Conclusions
25

Conclusions
 Our proposal approach for ArSL Recognition
 Achieve an excellent accuracy to identify ArSL from 2D images
 Robust against to rotation images with different angels and occluded
images horizontally or vertically.
 Robust against many previous ArSL approaches.
 Performance of this approach is measured by
 Using captured images with Matlab implementation
 Comparison with related work

Future Work
 Improving the results of in case of image
occlusion
 Increase the size of the dataset to check its
scalability.
 Identify characters from video frames and then
try to implement real time ArSL system.

Thanks Acknowledgement to By the respected co-authers
Abul Ella Hassenian3,4, M. K. Shahin 1, Basma Refaat 1
 1Faculty of Eng. Suez Canal University, Ismailia, Egypt
 2Faculty of Computers & Informatics , Suez Canal University, Ismailia, Egypt
 3Faculty of Computers and Information, Cairo University, Egypt
 4Scientic Research Group in Egypt (SRGE), http://www.egyptscience.netSuez Canal University

Sift based arabic sign language recognition aecia 2014 –november17-19, addis ababa, ethiopia- tarek-gaber

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (10)

Similaire à Sift based arabic sign language recognition aecia 2014 –november17-19, addis ababa, ethiopia- tarek-gaber

Similaire à Sift based arabic sign language recognition aecia 2014 –november17-19, addis ababa, ethiopia- tarek-gaber (20)

Plus de Tarek Gaber

Plus de Tarek Gaber (15)

Dernier

Dernier (20)

Sift based arabic sign language recognition aecia 2014 –november17-19, addis ababa, ethiopia- tarek-gaber