This research project involves parsing human rights reports produced by the United States Government and rating the human practices for various countries based on the CIRI (Cingranelli-Richards) Human Rights Data Project dataset. The United States Human Rights Reports are annual reports that cover internationally recognized human rights practices in regards to individual, civil, political, and worker rights. CIRI analyzed the annual reports from 1981 to 2011 and then stopped releasing the dataset for any further years. CIRI coders relies on a manual process of scoring the Human Rights Reports and then applying rating scores to each human rights practice for each country. This project has automated the process of scoring the human rights country reports. To accomplish this objective, we use GATE (General Architecture for Text Engineering) text mining platform. GATE is an open source software project used to provide solutions for text processing. We used various customizable GATE plugins in conjunction with the coding schemes provided by the CIRI Project documentation to create an automated ratings process. The accuracy of this tool was evaluated by comparing the automated ratings to the existing ratings within the CIRI dataset. Evaluation results indicate that our automated process has higher accuracy (ranges from 60 to 85%) for physical integrity human practices whereas much lower accuracy for empowerment (ranges from 40% to 60%) and women’s rights (ranges from 30% to 60%) practices. CIRI scoring process for physical integrity consists of primarily quantified measures whereas other two practices has more of subjective measures. Thus, additional research and modifications to text mining algorithm is required to improve the automated process of rating human right reports.

1. Automating CIRI Ratings of
Human Rights Reports Using GATE
Joshua Joiner and Karthikeyan Umapathy
School of Computing,
University of North Florida,
Jacksonville, FL USA 32224
R E S E A R C H C O N T E X T
This project involves parsing human rights reports
produced by the U.S Government and rating the human
practices for various countries. The U.S Human Rights
Reports are annual reports that cover internationally
recognized human rights practices in regards to individual,
civil, political, and worker rights.
T E X T M I N I N G T O O L
GATE is an open source text mining platform used for
developing custom text processing solutions.
G E N E R A T I N G C I R I R A T I N G U S I N G G A T E
C O N C L U S I O N S
In conclusion, I believe the automated process will not
provide a high accuracy when comparing to the CIRI
dataset because the dataset was compiled by humans. I do,
however, believe that processes involved in creating the
automated process can create a more objective standard
when analyzing country report text and producing ratings
for the human practices. There also needs to be more
patterns implemented within the automated process to
more accurately match with the qualitative text from the
U.S Human Rights Reports.
Text Mining of Human Rights Reports
Project Objective:
Denmark Physical Integrity
U.S. Department of
State
CIRI coders rely on a manual process of reading through
the Human Rights Reports and then applying ratings to
each human rights practice for each country.
• The objective of this project is to automate the process
of scouring the human rights country reports.
CIRI (Cingranelli-Richards) Human Rights Data Project
rates the human rights practices of the U.S. Human Rights
country reports. Students, scholars, policymakers, and
analysts use the CIRI ratings for practical and research
purposes.
CIRI Rating of Human Rights Reports
Standard ANNIE process flow:
C I R I R A T I N G S C O M P A R I S O N
Denmark Empowerment Rights
E V A L U A T I O N P L A N C O N T R I B U T I O N S
• CIRI Coding Annotation Processing Resource
• Custom JAPE patterns for keywords and
phrases.
• Custom annotations for entity extraction.
• Custom implementation of sentiment
analysis.
• Database Storage
• CIRI Dataset Source Ratings.
• Automatically generated CIRI Ratings.
The F-Measure scores in the tables above show which
type of ratings the automated system correctly matches.
KILL TORT POLPRIS DISAP
FORMOV DOMMOV ELECSD ASSN
WORKER SPEECH NEW_RELFRE