Enhancing and Restoring Safety & Quality Cultures - Dave Litwiller - May 2024...
UAIC Participation at RTE4
1. UAIC Participation at RTE4 Adrian Iftene [email_address] „ Al. I. Cuza“ University, Iasi, Romania Faculty of Computer Science
2.
3.
4. System presentation Initial data Minipar module Final result NE Rule Negation Rules Acronyms Background knowledge eXtended Wordnet GATE VerbOcean Numbers & Dates Contradiction Rules In RTE-3 competition Two-way 69.13 % Three-way 54.7 % After RTE-3 competition Test Data 2007: Two-way 74.1 % Three-way 71.8 % Test Data 2008 : Two-way 72.1 % Three-way 68.5 % Fitness Rule Main module TreeTagger Resources DIRT Dependency trees for (T, H) pairs LingPipe module Named entities for (T, H) pairs Wikipedia
5. Example NE (N) subj loc adv (A) subj mod … … NE (N) loc … … DIRT, VerbOcean WordNet Acronyms, BK DIRT: solve=resolve 0.31453 DIRT: convict=arrest 0.28895 DIRT: convict=acquit 0.302455,OPPOSITE VerbOcean: increase<>decrease, VerbOcean: leave<>stay WordNet: trouble=problem WordNet: talk=discussion Acronym: EU=European Union BK: Buenos Aires [in] Argentine BK: 16 [is] sixteen Hypothesis Text verb H (V) noun (N) verb T (V) noun (N) … … … …
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18. Results in RTE4 68.50% 1000 685 1000 Total 54.25% 42.86% 73.89% 203 150 350 Unknown 58.72% 46.00% 81.18% 85 69 150 Contradiction 76.90% 93.20% 65.45% 712 466 500 Entailment F-measure Recall Precision Total offered by our system Correct offered by our system In Gold Answer Type 72.10% 1000 721 1000 Total 64.72% 51.00% 88.54% 288 255 500 No 76.90% 93.20% 65.45% 712 466 500 Yes F-measure Recall Precision Total offered by our system Correct offered by our system In Gold Answer Type
19. Comparison with RTE-3 To be able to see each component’s relevance, the systems were run in turn with each component removed 5.55 % 68.10 % - - Without Contradiction rule 7.21 % 66.90 % 16.71 % 57.58 % Without NE rule 4.72 % 68.70 % 2.17 % 67.63 % Without Negation rule 2.36 % 70.40 % 2.00 % 67.75 % Without BK 0.42 % 71.80 % 1.08 % 68.38 % Without Acronyms 4.16 % 69.10 % 1.63 % 68.00 % Without WordNet 0.97 % 71.40 % 0.54 % 68.76 % Without DIRT Relevance Precision Relevance Precision RTE-4 RTE-3 System Description
![UAIC Participation at RTE4 Adrian Iftene [email_address] „ Al. I. Cuza“ University, Iasi, Romania Faculty of Computer Science](https://image.slidesharecdn.com/tac2008rte4iftene-120201105419-phpapp02/85/UAIC-Participation-at-RTE4-1-320.jpg)
![Overview ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)