Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Understanding Online Socials Harm: Examples of Harassment and Radicalization

359 vues

Publié le

Abstract: As social media permeates our daily life, there has been a sharp rise in the misuse of social media affecting our society in large. Specifically, harassment and radicalization have become two major problems on social media platforms with significant implications on the well-being of individuals as well as communities. A 2017 Pew Research survey on online harassment found that 66% of adult Internet users have observed online harassment and 41% have personally experienced it. Nearly 18% of Americans have faced severe forms of harassment online such as physical threats, harassment over a sustained period, sexual harassment or stalking. Moreover, malicious organizations (e.g., terrorist groups, white nationalists not classified legally as terrorists but as a group with extreme ideology) have been using social media for sharing their propaganda and misinformation to persuade individuals and eventually recruit them to propagate their ideology. These communications related to harassment and radicalization are complex concerning their language and contextual characteristics, making recognition of such narratives challenging for researchers as well as social media companies. As most of the existing approaches fail to capture fundamental nuances in the language of these communications, two prominent challenges have emerged: ambiguity and sparsity. Sole data level bottom-up analysis has been unsuccessful in revealing the actual meaning of the content. Considering the significant sensitivity of these problems and its implications at individual and community levels, a potential solution requires reliable algorithms for modeling such communications.

Our approach to understanding communications between source and target requires deciphering the unique language, semantic and contextual characteristics, including sentiment, emotion, and intention. This context-aware and knowledge-enhanced computational approach to the analysis of these narratives breaks down this long-running and complex process into contextual building blocks that acknowledge inherent ambiguity and sparsity. Based on prior empirical and qualitative research in social sciences, particularly cognitive psychology, and political science, we model this process using a combination of contextual dimensions -- e.g., for Islamist radicalization: religion, ideology, and hate -- each elucidating a degree of radicalization and highlighting independent features to render them computationally accessible.

Publié dans : Médias sociaux
  • Login to see the comments

  • Soyez le premier à aimer ceci

Understanding Online Socials Harm: Examples of Harassment and Radicalization

  1. 1. Understanding Online Socials Harm: Examples of Harassment and Radicalization Prof. Amit Sheth Founding Director, AI Institute University of South Carolina AI @ UofSC 33rd Annual IFIP WG 11.3 Conference on Data and Applications Security and Privacy (DBSec'19) Charleston, SC, USA July 15 -17, 2019 Icons by thenounproject Slides by SlideModel
  2. 2. 2 The youngest adults stand out in their social media consumption 88% of 18- to 29-year-olds indicate that they use any form of social media. By Pew Research Center “Social Media Use Report 2018”
  3. 3. 3 Social Good and Social Harm on Social Media A spectrum to demonstrate the variety of social good to social harm Adapted from : Purohit, Hemant & Pandey, Rahul. (2019). Intent Mining for the Good, Bad, and Ugly Use of Social Web: Concepts, Methods, and Challenges. 10.1007/978-3-319-94105-9_1. Zika Virus Monitoring Help Fighting depression Disaster Relief Opioid Usage Monitoring Joking Marketing Sensationalizing Harassment Accusing Rumouring Deceiving Fake News Radicalization Illicit Drugs Social HarmSocial Good Positive Effects Negative Effects
  4. 4. 4 Fake-porn videos are being weaponized to harass and humiliate women: ‘Everybody is a potential target’ ‘Deepfake’ disturbingly realistic, computer-generated videos with photos taken from the Web, and ordinary women are suffering the damage
  5. 5. 5 Different meanings of diagnostic terms Ambiguity Different perceptions of same concepts. Subjectivity Low prevalence of relevant content Sparsity Nature of content with more than one context. Multi Dimensionality Significant implications in a big scale application. False Alarms Knowledge Graphs and Knowledge Networks: The Story in Brief - IEEE Internet Computing Magazine 2019 Multimodal Content Different modalities of data Challenges --Complex Problems
  6. 6. 6 1. Online Harassment Context in an interaction determines bad behavior.
  7. 7. 7 Severity of online harm can differ based on several criteria It can span for more than a decade in one’s life Or it can lead to teenage suicides Police accuse two students, age 12, of cyberbullying in suicide By Jamiel Lynch, CNN Teenage cyber bullying victims are Use-cases of Online Harassment
  8. 8. 9 Existing Approaches Tweet Content Binary Classifier Harassing Non-Harassing People Network Our Approach (Incorporating context) Tweet Content Multiclass Classifier Sexual harassment Appearance related Harassment Traditional Approach vs Recent Advances
  9. 9. 10 Problem Definition & Sparsity Dataset # of Tweets Classes (%) Waseem et al. (2016) 16093 Racism (12%), Sexism (19.6%), Neither (68.4%) Davidson et al. (2017) 24,802 Hate (5%), Non- hate (95%) Zhang et al. (2018) 2,435 Hate (17%), Non-hate (83%) Mostly binary classifications Datasets have small percentages of positive (harassing) instances A Quality Type-aware Annotated Corpus And Lexicon For Harassment Research [Rezvan et al.] This paper provides both a quality annotated corpus and an offensive words lexicon capturing different types of harassment content: (i) sexual (7.4%) (ii) racial (22.5%) (iii) appearance-related (21.8%) (iv) intellectual (26%) (v) political (22.4%) Harassing (12.9%) Non-Harassing (87.1%) Challenges in Online Harassment
  10. 10. 12 According to Pew research center (2017) Subjectivity “pEoPlE dO iT tO tHeMsElVeS” shut the f**k up you know what im not gonna argue anymore u guys are all so f**king ignorant when it comes to addiction so PLEASe stop f**king speaking on it all the girls who i have beef w are little ass girls who think they can say the n word but get scared when black guys come around 🤔 🤔 on that note, gn twit An interaction example from Highschool students’ tweet corpus Challenges in Online Harassment
  11. 11. “Language used to express hatred towards a targeted individual or group, or is intended to be derogatory, to humiliate, or to insult the members of the group, on the basis of attributes such as race, religion, ethnic origin, sexual orientation, disability, or gender is hate speech” - Founta et al. 2018 13 Challenges in Online Harassment Detection Ambiguity Researchers have defined harassment using jargon that overlaps, causing ambiguity in annotations “Profanity, strongly impolite, rude or vulgar language expressed with fighting or hurtful words in order to insult a targeted individual or group is offensive language ” - Founta et al. 2018 Ex. 1: @user_name nah you just a dumb hoe who doesn’t know her place 😂 😂 This tweet belongs to hate speech and offensive language based on above definitions Examples from Highschool students’ tweet corpus Challenges in Online Harassment Ex. 2: IS THAT A MICROAGRESSION AGAINST MEXICANS BY STEREOTYPING THEM AS ILLEGALS?!? only if you were vegan you wouldn’t be such a racist pig This tweet falls into the category of hate speech but not necessarily offensive language
  12. 12. 14 What? How? Who? What causes online harassment? [REASON] How can online harassment happen? [METHOD] What are the effects of online harassment? [RESULT] Victim Appearance/ Religion/ Race etc. Harasser Xenophobia/ Homophobia/ Intolerance etc. Victim feels: ~ Offended ~ Discriminated ~ Afraid of losing life or losing social capital ~ Depressed ~ Suicidal Direct vs. Indirect Where? ~ Flaming (the act of posting or sending offensive messages over the Internet) ~ Doxxing (broadcasting private or identifying information of individuals) ~ Dogpiling (several people in twitter addressing someone, usually negatively in a short period of time) ~ Impersonation ~ Public shaming ~ Threats [ACTORS] ~ Harasser ~ Victim ~ Bystanders: a. Aggravators (People who try to fuel a harassing situation indirectly, for example by retweeting harassing tweets) a. Empathizers (People who empathize with the victim, providing support) ~ Social Media (Twitter, Facebook, Instagram) ~ Discussion Boards (Reddit, 4chan) ~ Email ~ Private Messaging ~ Online Gaming * Policies of platforms [PLATFORM] Frequently vs. One- time Cyber Bullying vs. Cyber Aggression Online Harassment - Dimensions
  13. 13. 16 Resolving Data Scarcity Issue Using Generative adversarial networks (GANs) to generate text, increasing the positive(harassing) examples in a dataset Changing the generator objective function in the GAN to incorporate domain specific knowledge Multiclass classification of harassing tweets Harassment type prediction was done using a multiclass classifier The process of tweet vectorization leveraged domain knowledge in the form of an offensive words lexicon Current Research Directions Pursued
  14. 14. 17 Key Takeaways ● In spite of recognition of importance, the problem is still not well-understood, and not well-defined. ● Use cases and data show that the problem is far more complex and nuanced. Understanding context is critical. ● Current social media platforms appear to have little or no automated processes for detection and prevention. Oversimplification of problem definition largely relying on machine learning without significant domain specific knowledge has rendered solutions practically useless.
  15. 15. 18 2. Online Extremist Communications Islamist Extremism and White Supremacy
  16. 16. 19 Challenges in Online Harassment Detection Efforts by High-Tech Companies Capabilities of social media companies (Twitter, Facebook and Google) are inadequate and ineffective. Governments insisted that the industry had a ‘social responsibility’ to do more to remove harmful content. If unsolved, social media platforms will continue to negatively impact the society. Unsolved: Detection of Extremism on Social Media
  17. 17. 20 Challenges in Online Harassment Detection ● One thousand Americans between 1980 and 2011. 300 Americans since 2011 attempted or traveled. ● > 5 thousand individuals from Europe traveled to Join Extremist Terrorist Groups (ISIS, Al-Qaeda) abroad through 2015, ● Most inspired and persuaded online. “The Travelers” *George Washington University, Program on Extremism
  18. 18. ● 24 year old college student from Alabama became radicalized on Twitter. After a year, moved to Syria to join ISIS. ● Self-taught, she read verses from the Qur’an, but interpreted them with others in the extremist network. ● Persuaded that when the true Islamic State is declared, it is obligatory to do hijrah, which they see as the pilgrimage to ’the State’. 21 Illustrative Case *New York Times: “Alabama Woman Who Joined ISIS Can’t Return Home, U.S. Says”
  19. 19. 22 Challenges in Online Harassment DetectionRadicalization Scale (Achilov et al.) 0 None Mainstream religious views and orientations Indicator: Islam; Allah; jihad (self struggle); halal; democracy, islam, salah, fatwa, hajj. 1 Low Attitudinal support for politically moderate Islamism Indicator: Hadith; Caliphate (Khilafah) justified; Sharia better (than secular law); Hypocrisy west. 2 Elevated Emergent support for exclusive rule of the Shari’a law Indicator: Shariah best; revenge (justified); jihad (against West); justify Daesh (ISIS) 3 High Support for extremist networks and travel to “Darul Islam” Indicator: Kafir; infidel; hijrah to Darul- Islam; (supporting) fatwa Al- Awlaki; mushrikeen. 4 Severe Call for action to join the fight and the use of violence. Indicator: apostate; sahwat; taghut; kill; kafir; kuffar; murtadd; tawaghit; al_baghdadi; martyrdom khilafah
  20. 20. 23 Challenges in Online Harassment DetectionRadicalization Process over time Ultimately, analysis of content in context will provide better finer-granular understanding the underlying factors in the radicalization process. Non-extremist ordinary individual Radicalized extremist individual 0 1 2 4 SevereHighLowNon e Elevated 3
  21. 21. Islamist Extremism on Social Media (e.g., recruiter, follower) with respect to different stages of radicalization. Modeling users psychological process over a time period. Persuasive relevant to Islamist extremism. Domain Knowledge of the context (“jihad” has different meaning in different context) Multidimensionality Radicalization
  22. 22. Security Implications Specifically, unfair classification of non- extremist individuals as extremist. False alarm might potentially impact millions of innocent people. 25 Local and Global security implications, while predicting online terrorist activities and involved individuals.
  23. 23. 26 Multidimensionality of Extremist Content ● Dimensions to define the context: ○ Based on literature and our empirical study of the data, three contextual dimensions are identified: Religion, Ideology, Hate ● The distribution of prevalent terms (i.e., words, phrases, concepts) in each dimension is different. ● These terms should be represented in different dimensions, to disambiguate especially diagnostic terms (e.g., jihad): .
  24. 24. Extremist Content 27 Prevalent Key Phrases Prevalent Topics isis, syria, kill, iraq, muslim, allah, attack, break, aleppo, assad, islamicstate, army, soldier, cynthiastruth, islam, support, mosul, libya, rebel, destroy, airstrike Caliphate_news, islamic_state, iraq_army, soldier_kill, iraqi_army, syria_isis, syria_iraq, assad_army, terror_group, shia_militia, isis_attack, aleppo_syria, martyrdom_operation, ahrar_sham, assad_regime, follow_support, lead_coalition, turkey_army, isis_claim, kill_isis Imam_anwar_awlaki, video_message_islamicstate, fight_islamic_state, isisclaim_responsibility_attack, muwahideen_powerful_middleeast, isis_tikrit_tikritop, amaqagency_islamicstate_fighter, sinai_explosion_target, alone_state_fighter, intelligence_reportedly_kill, khilafahnew_islamic_state, yemanqaida_commander_kill, isis_militant_hasakah, breakingnew_assad_army, isis_explode_middle, hater_trier_haleemah, trust_isis_tighten, qamishlus_isis_fighting, defeat_enemy_allah, kill_terrorist_baby, ahrar_sham_leader islamic state, syria, isis, kill, allah, video, minute propaganda video scenes, jaish islam release, restock missile, kaffir, join isis, aftermath, mercy, martyrdom operation syrian opposition, punish libya isis, syria assad, islam sunni, swat, lose head, wilayatalfurat, somali, child kill, takfir, jaish fateh, baghdad, iraq, kashmir muslim, capture, damascus, report rebel, british, qala moon, jannat, isis capture, border cross, aleppo, iranian soldier, tikrit tikrittop, lead shia military kill, saleh abdeslam refuse cooperate Green: Religion Blue: Ideology Red: Hate Corpus: 538 Twiter verified extremists, 48K tweets
  25. 25. 28 “Reportedly, a number of apostates were killed in the process. Just because they like it I guess.. #SpringJihad #CountrysideCleanup” “Kindness is a language which the blind can see and the deaf can hear #MyJihad be kind always” “By the Lord of Muhammad (blessings and peace be upon him) The nation of Jihad and martyrdom can never be defeated” “Jihad” can appear in tweets with different meanings in different dimensions of the context. H I R Example Tweets with “Jihad”
  26. 26. 29 Challenges in Online Harassment Detection ● Same term can have different meanings for each dimensions. ● Example: “Meaning of Jihad” is different for extremists and non-extremists. ○ For extremists, meaning closer to “awlaki”, “islamic state”, “aqeedah” For non-extremists, closer to “muslims”, “quran”, “imams” Ambiguity of Diagnostic terms/phrases ExtremistsNon-Extremists
  27. 27. ● Different Contextual Dimensions incorporating: ○ Knowledge Graphs ○ Dimension Corpora ● Utilization of Deep Learning models, generate knowledge-enhanced representations ● KG creation: Religion: Qur’an, Hadith Ideology: Books, lectures of ideologues [Not KG: Hate: Hate Speech Corpuss (Davidson et al. 2017)] ● Can be applied over many social problems. 30 Modeling Modeling Modeling Dimension 1 Dimension 2 Dimension 3 DimensionDimensionDimension Dimension Modeling Process Dimension based Knowledge enhanced Representation Contextual Dimension Modeling
  28. 28. (Hate) Capturing similarity: ● Learning word similarities from a substantial knowledge graph ● A solution via distance between concepts in the knowledge graph. Modeling 31 Using a Knowledge Graph “You shall know a word by the company it keeps” (J. R. Firth 1957: 11)
  29. 29. Capturing similarity (and resolving ambiguity): ● Learning word similarities from a large corpora. ● A solution via distributional similarity-based representations. Modeling 32 (Hate) Using a Corpus “You shall know a word by the company it keeps” (J. R. Firth 1957: 11)
  30. 30. 33 ● Found two distinct groups employing different contexts with different density. ● Religion and Hate are usually mixed, suggesting that extremists might employ different hate tactics. ● A small group of users employ ideological context far more often than others, suggesting these users might be disseminators of ideologically intense content. Density of Dimensions in Extremist Content
  31. 31. 34 ● Tri-dimension model performs best. ● Precision used as metric, to emphasize reduction on misclassification of non-extremist content. ● Implications in a large scale application. Results
  32. 32. ● False alarms: significantly reduced via incorporation of three specific dimensions of context. ● Extremist users employ religion along with hate, suggesting they employ different hate tactics for their targets. ● Inclusion of all three contextual dimensions significantly reduces the likelihood of an unfair mistreatment towards non-extremist individuals, in a real world application. ● Each dimension plays different roles in different levels of radicalization, capturing nuances as well as linguistic and semantic cues better throughout the radicalization process. 35 Key Insights
  33. 33. 36 Public/ Society Social Interactions Cognitive Neuro Cognitive Process ● Human brain processes information from extremist narratives on social media, that includes different contexts, emotions, sentiment, etc. ● Individuals change behavior, make choices in consuming/sharing content with an intent. ● Coordination, information flow and diffusion on social networks. ● Outcomes/impact on society through events and collective actions (eg, civil war or result of an election). Our Highly Multidisciplinary Approach
  34. 34. 37 Weaponized Ambiguity Sparsity Complexity
  35. 35. 38 Context-Aware Harassment Detection on Social Media(wiki link) is an interdisciplinary project among the Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis), the Department of Psychology, and Center for Urban and Public Affairs (CUPA) at Wright State University. We are supported by the NSF Award#: CNS 1513721 Supporting Grants
  36. 36. 39 Thank You! Special Thanks: Ugur Kursuncu and Thilini Wijesiriwardene
  37. 37. 1. Hinduja, S. and Patchin, J.W., 2010. Bullying, cyberbullying, and suicide. Archives of suicide research, 14(3), pp.206-221 2. Rezvan, M., Shekarpour, S., Balasuriya, L., Thirunarayan, K., Shalin, V.L. and Sheth, A., 2018, May. A Quality Type-aware Annotated Corpus and Lexicon for Harassment Research. In Proceedings of the 10th ACM Conference on Web Science (pp. 33-36). ACM. 3. Zeerak Waseem. Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proc. of the Workshop on NLP and Computational Social Science, pages 138–142. Association for Computational Linguistics, 2016. 4. Thoams Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. In Proceedings of the 11th Conference on Web and Social Media. AAAI, 2017. 5. Zhang, Ziqi & Luo, Lei. (2018). Hate Speech Detection: A Solved Problem? The Challenging Case of Long Tail on Twitter. Semantic Web. Accepted. 10.3233/SW-180338. 40 References