A Comparative Study On Featuree Selection In Text2
•Télécharger en tant que PPTX, PDF•
0 j'aime•335 vues
Recommandé
Contenu connexe
Similaire à A Comparative Study On Featuree Selection In Text2
Similaire à A Comparative Study On Featuree Selection In Text2 (20)
Plus de Trector Rancor
Dernier
Dernier (20)
A Comparative Study On Featuree Selection In Text2
- 1. A Comparative Study on Featuree Selection in Text Categorization Presented by Hector Franco TCD
- 2. objective • Reduce the number of dimensions. Some methods have problems with too high dimension.
- 3. Statistical classification methods. 1. Regression models 2. Knn 3. Bayes 4. Decision treees 5. Neural netwoks 6. Symbolic rule learning 7. Inductive learning algorithms
- 4. Features: • DF Document frequency thresholding • IG Information Gain • MI Mutual information • CHI statistic • TS Term strength
- 5. DF Document frequency thresholding • Number of documents in which term occurs. • It remove rare terms.
- 6. Information gain • Of the term t: • Time: O(N) space O(VN) • N=Documents, V=vocabulary
- 7. Mutual information • If t and c indpendent -> value 0. O(VN)
- 8. Statistic (CHI) • Measure of the lack of independence between t and c, • A t and c occurs, B t and not c • C not t and c , D not t and not c • N total number of documents It t and c independent value =0.
- 9. Statistic (CHI) O(VN)
- 10. Ts term strength • Based on document clustering • How common is a term is likely to appear in closely related documents. • O(N^2)
- 11. EXPERIMENTS • Classifiers – kNN – LLSF • Corporas: – Reuters-22173 – OHSUMED • Use of SMART system for unified preprocessing.
- 12. Reduction on number of words Have the best performance at 2000 vocabulary size Best ig (more reduction)and chi
- 14. Creative commons license You are free: •to copy, distribute, display, and perform the work •to make derivative works Under the following conditions: •Attribution. You must give the original author credit. What does quot;Attribute this workquot; mean? The page you came from contained embedded licensing metadata, including how the creator wishes to be attributed for re-use. You can use the HTML here to cite the work. Doing so will also include metadata on your page so that others can find the original work as well. •Non-Commercial. You may not use this work for commercial purposes. •For any reuse or distribution, you must make clear to others the licence terms of this work. •Any of these conditions can be waived if you get permission from the copyright holder. •Nothing in this license impairs or restricts the author's moral rights.