Datasets:
whoTarget
stringclasses 3
values | intentYN
stringclasses 4
values | sexYN
stringclasses 3
values | sexReason
stringlengths 0
199
| offensiveYN
stringclasses 4
values | annotatorGender
stringclasses 6
values | annotatorMinority
stringlengths 0
70
| sexPhrase
stringlengths 0
265
| speakerMinorityYN
stringclasses 4
values | WorkerId
stringlengths 16
20
| HITId
stringlengths 30
30
| annotatorPolitics
stringclasses 8
values | annotatorRace
stringclasses 8
values | annotatorAge
stringlengths 0
5
| post
stringlengths 2
1.22k
| targetMinority
stringlengths 0
124
| targetCategory
stringclasses 8
values | targetStereotype
stringlengths 0
170
| dataSource
stringclasses 12
values |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0.0 | 0.66 | 0.0 | 1.0 | woman | -8935932304856669427 | 363A7XIFV4G2799C5V96YERJA9AVAM | liberal | white | 45.0 | RT @_LexC__: I'm convinced that some of y'all bitches get pregnant purposely because "birth control & plan b pills" are effective 😕🐸☕️ | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 0.5 | man | 6347880360297734464 | 363A7XIFV4G2799C5V96YERJA9AVAM | mod-liberal | white | 35.0 | RT @_LexC__: I'm convinced that some of y'all bitches get pregnant purposely because "birth control & plan b pills" are effective 😕🐸☕️ | t/davidson |
|||||||
0.0 | 0.33 | 0.0 | 0.5 | man | -7452610791699819066 | 363A7XIFV4G2799C5V96YERJA9AVAM | liberal | asian | 23.0 | RT @_LexC__: I'm convinced that some of y'all bitches get pregnant purposely because "birth control & plan b pills" are effective 😕🐸☕️ | t/davidson |
|||||||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | -500114755446676507 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 25.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | all stupid | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | -500114755446676507 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 25.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | are not people but apes. | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.0 | 7912096326098817047 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 32.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | black people are monkeys | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.0 | 7912096326098817047 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 32.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | black people are from the jungles | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.0 | 7912096326098817047 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 32.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | using racial slurs is okay | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | gender, sexuality, mental illness | 0.0 | 118631008378348260 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 28.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | black folks are animals | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | woman | gender, sexuality, mental illness | 0.0 | 118631008378348260 | 3JTPR5MTZS6RLS3JBV4IOU0G2X35K5 | liberal | white | 28.0 | RT @iBeZo: Stupid fucking nigger LeBron. You flopping stupid jungle bunny monkey faggot. | black folks | race | black folks are stupid | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | 1.0 | 2430897476320006833 | 3OKP4QVBP2RJPCJHG0AKYDO9NUNAGD | cons | white | 32.0 | I need a girl from Jamaica I can't fuck with these basic black bitches | black folks | race | black women are basic | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | man | 1.0 | 2430897476320006833 | 3OKP4QVBP2RJPCJHG0AKYDO9NUNAGD | cons | white | 32.0 | I need a girl from Jamaica I can't fuck with these basic black bitches | black folks | race | t/davidson |
||||
1.0 | 1.0 | 0.0 | 1.0 | man | white, male | 0.0 | 1389628790122253253 | 3WGCNLZJKF2KJ4WQIRGJHRS9681D1I | cons | white | 31.0 | Molly make bitches crazy | women | gender | women can't handle drugs | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | white, male | 0.0 | 1389628790122253253 | 3WGCNLZJKF2KJ4WQIRGJHRS9681D1I | cons | white | 31.0 | Molly make bitches crazy | women | gender | women are crazy | t/davidson |
||
0.0 | 0.0 | 0.0 | woman | 6927542478311092738 | 3HRWUH63QUWSTYOM88WYGMHKWS85NA | liberal | white | 24.0 | RT @rihanna: I ain't got no type...bad bitches is the only thing that I like | t/davidson |
||||||||
0.33 | 0.0 | 0.0 | woman | women | -6837958490067487319 | 3HRWUH63QUWSTYOM88WYGMHKWS85NA | liberal | white | 34.0 | RT @rihanna: I ain't got no type...bad bitches is the only thing that I like | t/davidson |
|||||||
0.0 | 0.0 | 0.0 | woman | none | -4903257119943710595 | 3HRWUH63QUWSTYOM88WYGMHKWS85NA | liberal | white | 32.0 | RT @rihanna: I ain't got no type...bad bitches is the only thing that I like | t/davidson |
|||||||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.5 | 8789702980570783632 | 3P888QFVX3OZ457WTFKETMCGIYZQOM | other | hisp | 27.0 | This niggga said I be branding bitches 😹😂😂😂😂😏 | black folks | race | black folks brand women | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.5 | 8789702980570783632 | 3P888QFVX3OZ457WTFKETMCGIYZQOM | other | hisp | 27.0 | This niggga said I be branding bitches 😹😂😂😂😂😏 | black folks | race | black folks think they own women | t/davidson |
|||
1.0 | 0.66 | 0.0 | 0.5 | man | 0.0 | 4554125108380068779 | 3P888QFVX3OZ457WTFKETMCGIYZQOM | mod-liberal | white | 27.0 | This niggga said I be branding bitches 😹😂😂😂😂😏 | women | gender | women are property to some men | t/davidson |
|||
1.0 | 0.66 | 0.0 | 0.5 | man | 0.0 | 4554125108380068779 | 3P888QFVX3OZ457WTFKETMCGIYZQOM | mod-liberal | white | 27.0 | This niggga said I be branding bitches 😹😂😂😂😂😏 | women | gender | women get "branded" by a man | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.0 | -8810733183869606469 | 35O6H0UNLSAWA8X1TA46NF03N835J6 | liberal | white | 34.0 | It's funny when u get a black female mad they start acting ghetto then a bitch | black folks | race | black women are trashy. | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.0 | -8810733183869606469 | 35O6H0UNLSAWA8X1TA46NF03N835J6 | liberal | white | 34.0 | It's funny when u get a black female mad they start acting ghetto then a bitch | black folks | race | black women are easily angered. | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | female | 0.0 | 5330619308534303933 | 35O6H0UNLSAWA8X1TA46NF03N835J6 | mod-cons | white | 35.0 | It's funny when u get a black female mad they start acting ghetto then a bitch | black women | gender | are loud and mean | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | woman | female | 0.0 | 5330619308534303933 | 35O6H0UNLSAWA8X1TA46NF03N835J6 | mod-cons | white | 35.0 | It's funny when u get a black female mad they start acting ghetto then a bitch | black women | gender | are all from the ghetto | t/davidson |
||
0.0 | 0.66 | 0.0 | 1.0 | man | -7452610791699819066 | 36FQTHX3Z3LNEACLRZ3WAUJ61733BT | liberal | asian | 23.0 | RT @_danieebro: all these bitches fake if you ask me | t/davidson |
|||||||
0.0 | 1.0 | 0.0 | 1.0 | man | -2877140746380073212 | 36FQTHX3Z3LNEACLRZ3WAUJ61733BT | cons | white | 42.0 | RT @_danieebro: all these bitches fake if you ask me | t/davidson |
|||||||
0.0 | 0.0 | 0.0 | man | 3541686222721437796 | 36FQTHX3Z3LNEACLRZ3WAUJ61733BT | liberal | white | 37.0 | RT @_danieebro: all these bitches fake if you ask me | t/davidson |
||||||||
0.0 | 1.0 | 0.0 | 1.0 | woman | -5471608738051449110 | 3L21G7IH47QNHFRHEN3XUC2AOW6Y1A | other | white | 35.0 | RT @__nijel: Niggas be out here loving ALL they hoes...that's a no no. | t/davidson |
|||||||
0.0 | 1.0 | 0.0 | 1.0 | woman | 2428559182657536645 | 3L21G7IH47QNHFRHEN3XUC2AOW6Y1A | liberal | white | 29.0 | RT @__nijel: Niggas be out here loving ALL they hoes...that's a no no. | t/davidson |
|||||||
0.33 | 0.0 | 0.0 | woman | women | -1563458821535984332 | 3L21G7IH47QNHFRHEN3XUC2AOW6Y1A | liberal | white | 22.0 | RT @__nijel: Niggas be out here loving ALL they hoes...that's a no no. | t/davidson |
|||||||
1.0 | 0.66 | 0.0 | 1.0 | woman | 0.5 | 980292540985390308 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | liberal | white | 51.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | conservatives | social | conservatives hate black people. | t/davidson |
|||
1.0 | 0.66 | 0.0 | 1.0 | woman | 0.5 | 980292540985390308 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | liberal | white | 51.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | conservatives | social | all conservatives hate obama because he is black. | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | Woman, African American | 0.0 | 7067515179960755087 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | mod-liberal | black | 30.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | black folks | race | nigger is a derogatory term for black people | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | woman | Woman, African American | 0.0 | 7067515179960755087 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | mod-liberal | black | 30.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | black folks | race | black people were called niggers | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | white, male | 0.0 | 1389628790122253253 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | cons | white | 31.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | black folks | race | said niggas | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | white, male | 0.0 | 1389628790122253253 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | cons | white | 31.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | black folks | race | republicans don't like blacks | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | white, male | 0.0 | 1389628790122253253 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | cons | white | 31.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | republicans | social | all republicans don't like black people | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | white, male | 0.0 | 1389628790122253253 | 3BO3NEOQM0BXL3PC6OANJP1P27FAI5 | cons | white | 31.0 | A guy claiming to be Republican called C-Span news and said "Republicans hate that nigger Obama" https://t.co/XZe0u7guZM | republicans | social | republicans are racist. | t/davidson |
||
0.0 | 0.66 | 0.0 | 1.0 | man | -7452610791699819066 | 3VW0145YLY6UIFI7L9LS4HBP93TMJD | liberal | asian | 23.0 | I don't love you hoes. | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 0.5 | man | -7790343187498138255 | 3VW0145YLY6UIFI7L9LS4HBP93TMJD | cons | white | 49.0 | I don't love you hoes. | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 0.5 | woman | 1869048359460155317 | 3VW0145YLY6UIFI7L9LS4HBP93TMJD | mod-cons | white | 22.0 | I don't love you hoes. | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 1.0 | man | -2877140746380073212 | 3L55D8AUFAR74I1FUKSHB8L1STYCYF | cons | white | 42.0 | RT @t_grun: Yeah I checked my connection, I have fucking wifi. Did you check if you were being a lil bitch nigga? http://t.co/0kvF0REwjd | t/davidson |
|||||||
0.0 | 1.0 | 0.0 | 1.0 | woman | 1869048359460155317 | 3L55D8AUFAR74I1FUKSHB8L1STYCYF | mod-cons | white | 22.0 | RT @t_grun: Yeah I checked my connection, I have fucking wifi. Did you check if you were being a lil bitch nigga? http://t.co/0kvF0REwjd | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 1.0 | man | -7713443391178782699 | 3L55D8AUFAR74I1FUKSHB8L1STYCYF | mod-liberal | white | 27.0 | RT @t_grun: Yeah I checked my connection, I have fucking wifi. Did you check if you were being a lil bitch nigga? http://t.co/0kvF0REwjd | t/davidson |
|||||||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | -3579354685786976851 | 3SA4EMRVJVWT1CEYRH3E3GQOFZY0PG | mod-cons | white | 35.0 | @JustDoItSlow on sale ass hoes | women | gender | women are sex objects | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | -3579354685786976851 | 3SA4EMRVJVWT1CEYRH3E3GQOFZY0PG | mod-cons | white | 35.0 | @JustDoItSlow on sale ass hoes | women | gender | women are refereed to as a derogatory word | t/davidson |
|||
0.0 | 0.0 | 0.0 | woman | none | -4903257119943710595 | 37MQ8Z1JQEQU609DXR3BZVCFG2KY2R | liberal | white | 32.0 | Up too early the damn birds are still chirping smh. | t/davidson |
|||||||
0.0 | 0.33 | 0.0 | 1.0 | woman | -4671295340405687436 | 37MQ8Z1JQEQU609DXR3BZVCFG2KY2R | other | white | 31.0 | Up too early the damn birds are still chirping smh. | t/davidson |
|||||||
0.0 | 0.0 | 0.0 | woman | 2428559182657536645 | 37MQ8Z1JQEQU609DXR3BZVCFG2KY2R | liberal | white | 29.0 | Up too early the damn birds are still chirping smh. | t/davidson |
||||||||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | 4554125108380068779 | 3RIHDBQ1NESYR9NHX9I7S4VAWBPHM3 | mod-liberal | white | 27.0 | The emoji I use when I'm flirting wit bitches ---> 😈 | women | gender | degrades women by calling them names | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | 4554125108380068779 | 3RIHDBQ1NESYR9NHX9I7S4VAWBPHM3 | mod-liberal | white | 27.0 | The emoji I use when I'm flirting wit bitches ---> 😈 | women | gender | women are called bitches | t/davidson |
|||
0.33 | 0.0 | 0.0 | woman | women | -7090297418013232722 | 30QQTY5GMKEKBSMET2NHS1NAVUW7UJ | other | white | 38.0 | I respect any nigga that tell me he got a bitch , so I fall the fuck back no hating 💯 | t/davidson |
|||||||
0.0 | 0.0 | 0.0 | 0.5 | woman | 1869048359460155317 | 30QQTY5GMKEKBSMET2NHS1NAVUW7UJ | mod-cons | white | 22.0 | I respect any nigga that tell me he got a bitch , so I fall the fuck back no hating 💯 | t/davidson |
|||||||
0.0 | 0.0 | 0.0 | man | -7790343187498138255 | 30QQTY5GMKEKBSMET2NHS1NAVUW7UJ | cons | white | 49.0 | I respect any nigga that tell me he got a bitch , so I fall the fuck back no hating 💯 | t/davidson |
||||||||
0.0 | 1.0 | 0.0 | 1.0 | woman | 2428559182657536645 | 36818Z1KV37IV07TLL30FJZ0JPI3AY | liberal | white | 29.0 | RT @Soy_Brent: steal a nigga's bitch, don't steal a nigga's tweets. | t/davidson |
|||||||
0.0 | 1.0 | 0.0 | 1.0 | woman | 6927542478311092738 | 36818Z1KV37IV07TLL30FJZ0JPI3AY | liberal | white | 24.0 | RT @Soy_Brent: steal a nigga's bitch, don't steal a nigga's tweets. | t/davidson |
|||||||
0.0 | 0.0 | 0.0 | woman | none | -4903257119943710595 | 36818Z1KV37IV07TLL30FJZ0JPI3AY | liberal | white | 32.0 | RT @Soy_Brent: steal a nigga's bitch, don't steal a nigga's tweets. | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 0.5 | man | -7790343187498138255 | 371Q3BEXDH3NGP0GNSK04KYLKQNSZF | cons | white | 49.0 | RT @caseytu88662466: Y these niggas on this hoe shit | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 0.5 | woman | 1869048359460155317 | 371Q3BEXDH3NGP0GNSK04KYLKQNSZF | mod-cons | white | 22.0 | RT @caseytu88662466: Y these niggas on this hoe shit | t/davidson |
|||||||
0.0 | 1.0 | 1.0 | refers to lots of sexual activity | 1.0 | woman | women | hoe shit | 7162172225567546910 | 371Q3BEXDH3NGP0GNSK04KYLKQNSZF | liberal | white | 27.0 | RT @caseytu88662466: Y these niggas on this hoe shit | t/davidson |
||||
0.0 | 1.0 | women wanting sex | 0.0 | woman | hoes bent over | 6927542478311092738 | 3VGET1QSZ0TX3WBJO22IAP87GA77WI | liberal | white | 24.0 | 🙌 RT @iDO_me2: this vodka be having all the hoes bent over😩💃 | t/davidson |
||||||
0.0 | 1.0 | 1.0 | refers to someone that's sexually promiscuous | 1.0 | woman | Woman, African American | hoes | 7067515179960755087 | 3VGET1QSZ0TX3WBJO22IAP87GA77WI | mod-liberal | black | 30.0 | 🙌 RT @iDO_me2: this vodka be having all the hoes bent over😩💃 | t/davidson |
||||
0.0 | 0.66 | 0.0 | 0.5 | man | -7790343187498138255 | 3VGET1QSZ0TX3WBJO22IAP87GA77WI | cons | white | 49.0 | 🙌 RT @iDO_me2: this vodka be having all the hoes bent over😩💃 | t/davidson |
|||||||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | 2385617559681638227 | 3P7RGTLO6E7ORYF06RVZW5G2UB4AKG | liberal | white | 26.0 | RT @Tahrell: Now, bitches gonna think just because they're married, they're better than every female that's not. Smh | women | gender | women are bitches | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | 2385617559681638227 | 3P7RGTLO6E7ORYF06RVZW5G2UB4AKG | liberal | white | 26.0 | RT @Tahrell: Now, bitches gonna think just because they're married, they're better than every female that's not. Smh | women | gender | women only want to get married | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | women | 0.0 | 2659038022143379063 | 37OPIVELUUXXP8ES06S2JD0A2U0AHC | mod-cons | white | 47.0 | RT @TheBloodShow: Ugly hoes be the loudest .. u need to duck off & hide yourself frfr | women | gender | women are loud if they are not good looking | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | woman | women | 0.0 | 2659038022143379063 | 37OPIVELUUXXP8ES06S2JD0A2U0AHC | mod-cons | white | 47.0 | RT @TheBloodShow: Ugly hoes be the loudest .. u need to duck off & hide yourself frfr | women | gender | women need to hide if they unattractive | t/davidson |
||
1.0 | 0.66 | 0.0 | 1.0 | man | 0.0 | 4554125108380068779 | 37OPIVELUUXXP8ES06S2JD0A2U0AHC | mod-liberal | white | 27.0 | RT @TheBloodShow: Ugly hoes be the loudest .. u need to duck off & hide yourself frfr | ugly folks | body | ugly women are unwanted | t/davidson |
|||
1.0 | 0.66 | 0.0 | 1.0 | man | 0.0 | 4554125108380068779 | 37OPIVELUUXXP8ES06S2JD0A2U0AHC | mod-liberal | white | 27.0 | RT @TheBloodShow: Ugly hoes be the loudest .. u need to duck off & hide yourself frfr | ugly folks | body | ugly women should disappear | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.5 | -4671295340405687436 | 3PGQRAZX02ENBZQBR6IHQDPJEWJSYO | other | white | 31.0 | Fuck all these dirty hoes these bitches basic | women | gender | women are "hoes" | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.5 | -4671295340405687436 | 3PGQRAZX02ENBZQBR6IHQDPJEWJSYO | other | white | 31.0 | Fuck all these dirty hoes these bitches basic | women | gender | women are "bitches" | t/davidson |
|||
1.0 | 1.0 | 1.0 | alludes to rape | 1.0 | woman | none | molly'd | 0.5 | -3114316428557054065 | 33Q5P9PUSPG798BKQ6U0PFU3YN2CZ6 | mod-cons | white | 33.0 | bitch probably molly'd the 1st nigga that said good mornin' back to her | women | gender | women are defined by who has sex with them | t/davidson |
1.0 | 1.0 | 1.0 | alludes to rape | 1.0 | woman | none | molly'd | 0.5 | -3114316428557054065 | 33Q5P9PUSPG798BKQ6U0PFU3YN2CZ6 | mod-cons | white | 33.0 | bitch probably molly'd the 1st nigga that said good mornin' back to her | women | gender | women are sex objects | t/davidson |
1.0 | 1.0 | 1.0 | alludes to rape | 1.0 | woman | none | molly'd | 0.5 | -3114316428557054065 | 33Q5P9PUSPG798BKQ6U0PFU3YN2CZ6 | mod-cons | white | 33.0 | bitch probably molly'd the 1st nigga that said good mornin' back to her | black folks | race | black people are niggers | t/davidson |
1.0 | 1.0 | 1.0 | alludes to rape | 1.0 | woman | none | molly'd | 0.5 | -3114316428557054065 | 33Q5P9PUSPG798BKQ6U0PFU3YN2CZ6 | mod-cons | white | 33.0 | bitch probably molly'd the 1st nigga that said good mornin' back to her | black folks | race | black people only think about sex | t/davidson |
1.0 | 0.33 | 0.0 | 1.0 | woman | women | 1.0 | 7162172225567546910 | 3H6W48L9F4JM92FJKOWZABZSG4DWPE | liberal | white | 27.0 | @G_Anderson4 lol it's gonna be so fun got two niglets on the way | black folks | race | black people are niggers. | t/davidson |
||
1.0 | 0.33 | 0.0 | 1.0 | woman | women | 1.0 | 7162172225567546910 | 3H6W48L9F4JM92FJKOWZABZSG4DWPE | liberal | white | 27.0 | @G_Anderson4 lol it's gonna be so fun got two niglets on the way | black folks | race | black babies are niglets. | t/davidson |
||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | 6347880360297734464 | 3H6W48L9F4JM92FJKOWZABZSG4DWPE | mod-liberal | white | 35.0 | @G_Anderson4 lol it's gonna be so fun got two niglets on the way | black folks | race | black people are a waste | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | 6347880360297734464 | 3H6W48L9F4JM92FJKOWZABZSG4DWPE | mod-liberal | white | 35.0 | @G_Anderson4 lol it's gonna be so fun got two niglets on the way | black folks | race | black people are a waste | t/davidson |
|||
1.0 | 0.33 | 0.0 | 1.0 | woman | 0.5 | 1869048359460155317 | 3H6W48L9F4JM92FJKOWZABZSG4DWPE | mod-cons | white | 22.0 | @G_Anderson4 lol it's gonna be so fun got two niglets on the way | black folks | race | black folks are n*ggers | t/davidson |
|||
1.0 | 0.33 | 0.0 | 1.0 | woman | 0.5 | 1869048359460155317 | 3H6W48L9F4JM92FJKOWZABZSG4DWPE | mod-cons | white | 22.0 | @G_Anderson4 lol it's gonna be so fun got two niglets on the way | black folks | race | black folks are inferior to whites | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.5 | -4671295340405687436 | 322ZSN9Z5GE8SSP6RZZ0KTHYLEYT4T | other | white | 31.0 | U wondering why I'm not answering I'm wondering where my money at niggas act like bitches these days | black folks | race | black people can be called the n word. | t/davidson |
|||
1.0 | 1.0 | 0.0 | 1.0 | woman | 0.5 | -4671295340405687436 | 322ZSN9Z5GE8SSP6RZZ0KTHYLEYT4T | other | white | 31.0 | U wondering why I'm not answering I'm wondering where my money at niggas act like bitches these days | black folks | race | black people were slaves. | t/davidson |
|||
1.0 | 0.66 | 0.0 | 1.0 | man | 1.0 | 1558992385113521549 | 322ZSN9Z5GE8SSP6RZZ0KTHYLEYT4T | libert | white | 39.0 | U wondering why I'm not answering I'm wondering where my money at niggas act like bitches these days | black folks | race | blacks are in debt | t/davidson |
|||
1.0 | 0.66 | 0.0 | 1.0 | man | 1.0 | 1558992385113521549 | 322ZSN9Z5GE8SSP6RZZ0KTHYLEYT4T | libert | white | 39.0 | U wondering why I'm not answering I'm wondering where my money at niggas act like bitches these days | black folks | race | blacks act like women | t/davidson |
|||
0.0 | 0.66 | 0.0 | 0.5 | man | -7790343187498138255 | 33NKDW9FFXCI4DB5PO6FSUESQN4CX9 | cons | white | 49.0 | RT @MsKeShera_: these bitches be sending shots but be missing me ian been @'d yet you cant be dissing me! | t/davidson |
|||||||
0.0 | 0.66 | 0.0 | 1.0 | woman | racial minority, gender | 4224836053735053558 | 33NKDW9FFXCI4DB5PO6FSUESQN4CX9 | liberal | asian | 25.0 | RT @MsKeShera_: these bitches be sending shots but be missing me ian been @'d yet you cant be dissing me! | t/davidson |
||||||
0.0 | 1.0 | 0.0 | 1.0 | man | 0.0 | -5870881806826799680 | 33NKDW9FFXCI4DB5PO6FSUESQN4CX9 | liberal | white | 38.0 | RT @MsKeShera_: these bitches be sending shots but be missing me ian been @'d yet you cant be dissing me! | t/davidson |
||||||
1.0 | 0.66 | 1.0 | they're talking about having sex with minorities | 1.0 | woman | fuck exotic bitches! | 0.0 | 6650634677954316496 | 3UUSLRKAULXYUQBS77XSI1JM9U1D7F | libert | white | 25.0 | I just wanna get lifted & fuck exotic bitches! | women | gender | women are only good for sex | t/davidson |
|
1.0 | 0.66 | 1.0 | they're talking about having sex with minorities | 1.0 | woman | fuck exotic bitches! | 0.0 | 6650634677954316496 | 3UUSLRKAULXYUQBS77XSI1JM9U1D7F | libert | white | 25.0 | I just wanna get lifted & fuck exotic bitches! | women | gender | women deserve name calling | t/davidson |
|
0.0 | 0.0 | 0.0 | woman | 2428559182657536645 | 3BAWBGQGYLTKO7BEYSXMONPT9157V8 | liberal | white | 29.0 | people be shooting their guns around here late as fuck got me thinking they tryna shoot my black ass 4 jammin jiggaboo music | t/davidson |
||||||||
0.33 | 0.0 | 0.0 | woman | women | -1563458821535984332 | 3BAWBGQGYLTKO7BEYSXMONPT9157V8 | liberal | white | 22.0 | people be shooting their guns around here late as fuck got me thinking they tryna shoot my black ass 4 jammin jiggaboo music | t/davidson |
|||||||
0.0 | 0.33 | 0.0 | 1.0 | woman | -8935932304856669427 | 3BAWBGQGYLTKO7BEYSXMONPT9157V8 | liberal | white | 45.0 | people be shooting their guns around here late as fuck got me thinking they tryna shoot my black ass 4 jammin jiggaboo music | t/davidson |
|||||||
0.33 | 0.0 | 0.0 | man | 6164268409378511170 | 367O8HRHKG27DSCR2ZUAAOLCJVLS44 | libert | white | 49.0 | I'm flexin on these bitches all gold bottles no Bombay | t/davidson |
||||||||
0.0 | 1.0 | 0.0 | 1.0 | woman | 2428559182657536645 | 367O8HRHKG27DSCR2ZUAAOLCJVLS44 | liberal | white | 29.0 | I'm flexin on these bitches all gold bottles no Bombay | t/davidson |
|||||||
0.0 | 0.33 | 0.0 | 0.5 | man | -2877140746380073212 | 367O8HRHKG27DSCR2ZUAAOLCJVLS44 | cons | white | 42.0 | I'm flexin on these bitches all gold bottles no Bombay | t/davidson |
|||||||
1.0 | 1.0 | 1.0 | Sexual reference | 1.0 | man | ass sucked | 0.0 | 6164268409378511170 | 31HLTCK4BLP3H0MFRV1GU02M84NVGY | libert | white | 49.0 | @Arbogucci guys not supposed to get their ass sucked queer | gay men | gender | gay people are disgusting | t/davidson |
|
1.0 | 1.0 | 1.0 | Sexual reference | 1.0 | man | ass sucked | 0.0 | 6164268409378511170 | 31HLTCK4BLP3H0MFRV1GU02M84NVGY | libert | white | 49.0 | @Arbogucci guys not supposed to get their ass sucked queer | gay men | gender | gay people are unnatural | t/davidson |
|
1.0 | 1.0 | 1.0 | reference to oral sex | 1.0 | man | guys not supposed to get their ass sucked queer | 0.0 | 7891575995618716029 | 31HLTCK4BLP3H0MFRV1GU02M84NVGY | mod-liberal | white | 40.0 | @Arbogucci guys not supposed to get their ass sucked queer | gay men | gender | gay men are into weird sex | t/davidson |
Dataset Card for "social_bias_frames"
Dataset Summary
Warning: this document and dataset contain content that may be offensive or upsetting.
Social Bias Frames is a new way of representing the biases and offensiveness that are implied in language. For example, these frames are meant to distill the implication that "women (candidates) are less qualified" behind the statement "we shouldn’t lower our standards to hire more women." The Social Bias Inference Corpus (SBIC) supports large-scale learning and evaluation of social implications with over 150k structured annotations of social media posts, spanning over 34k implications about a thousand demographic groups.
Supported Tasks and Leaderboards
This dataset supports both classification and generation. Sap et al. developed several models using the SBIC. They report an F1 score of 78.8 in predicting whether the posts in the test set were offensive, an F1 score of 78.6 in predicting whether the posts were intending to be offensive, an F1 score of 80.7 in predicting whether the posts were lewd, and an F1 score of 69.9 in predicting whether the posts were targeting a specific group.
Another of Sap et al.’s models performed better in the generation task. They report a BLUE score of 77.9, a Rouge-L score of 68.7, and a WMD score of 0.74 in generating a description of the targeted group given a post as well as a BLUE score of 52.6, a Rouge-L score of 44.9, and a WMD score of 2.79 in generating a description of the implied offensive statement given a post. See the paper for further details.
Languages
The language in SBIC is predominantly white-aligned English (78%, using a lexical dialect detector, Blodgett et al., 2016). The curators find less than 10% of posts in SBIC are detected to have the AAE dialect category. The BCP-47 language tag is, presumably, en-US.
Dataset Structure
Data Instances
Each instance contains a post that may contain an offensive statement and annotated information concerning the nature of the offensive implication as well as the demographics of the annotator and origin of the post. See the Social Bias Frames dataset viewer to explore more examples.
default
- Size of downloaded dataset files: 6.32 MB
- Size of the generated dataset: 44.47 MB
- Total amount of disk used: 50.80 MB
An example of 'validation' looks as follows.
{
"HITId": "3Z8UJEJOCZ74QHXOTFP0JOOGD7039J",
"WorkerId": "7162172225567546910",
"annotatorAge": "27.0",
"annotatorGender": "woman",
"annotatorMinority": "women",
"annotatorPolitics": "liberal",
"annotatorRace": "white",
"dataSource": "t/davidson",
"intentYN": "1.0",
"offensiveYN": "1.0",
"post": "Every half-breed cop in America is trying to rape white women.",
"sexPhrase": "rape white women",
"sexReason": "rape",
"sexYN": "1.0",
"speakerMinorityYN": "0.0",
"targetCategory": "race",
"targetMinority": "mixed folks",
"targetStereotype": "mixed folks are rapists.",
"whoTarget": "1.0"
}
Data Fields
The data fields are the same among all splits.
default
- whoTarget: a string, ‘0.0’ if the target is a group, ‘1.0’ if the target is an individual, and blank if the post is not offensive
- intentYN: a string indicating if the intent behind the statement was to offend. This is a categorical variable with four possible answers, ‘1.0’ if yes, ‘0.66’ if probably, ‘0.33’ if probably not, and ‘0.0’ if no.
- sexYN: a string indicating whether the post contains a sexual or lewd reference. This is a categorical variable with three possible answers, ‘1.0’ if yes, ‘0.5’ if maybe, ‘0.0’ if no.
- sexReason: a string containing a free text explanation of what is sexual if indicated so, blank otherwise
- offensiveYN: a string indicating if the post could be offensive to anyone. This is a categorical variable with three possible answers, ‘1.0’ if yes, ‘0.5’ if maybe, ‘0.0’ if no.
- annotatorGender: a string indicating the gender of the MTurk worker
- annotatorMinority: a string indicating whether the MTurk worker identifies as a minority
- sexPhrase: a string indicating which part of the post references something sexual, blank otherwise
- speakerMinorityYN: a string indicating whether the speaker was part of the same minority group that's being targeted. This is a categorical variable with three possible answers, ‘1.0’ if yes, ‘0.5’ if maybe, ‘0.0’ if no.
- WorkerId: a string hashed version of the MTurk workerId
- HITId: a string id that uniquely identifies each post
- annotatorPolitics: a string indicating the political leaning of the MTurk worker
- annotatorRace: a string indicating the race of the MTurk worker
- annotatorAge: a string indicating the age of the MTurk worker
- post: a string containing the text of the post that was annotated
- targetMinority: a string indicating the demographic group targeted
- targetCategory: a string indicating the high-level category of the demographic group(s) targeted
- targetStereotype: a string containing the implied statement
- dataSource: a string indicating the source of the post (
t/...
: means Twitter,r/...
: means a subreddit)
Data Splits
To ensure that no post appeared in multiple splits, the curators defined a training instance as the post and its three sets of annotations. They then split the dataset into train, validation, and test sets (75%/12.5%/12.5%).
name | train | validation | test |
---|---|---|---|
default | 112900 | 16738 | 17501 |
Dataset Creation
Curation Rationale
The main aim for this dataset is to cover a wide variety of social biases that are implied in text, both subtle and overt, and make the biases representative of real world discrimination that people experience RWJF 2017. The curators also included some innocuous statements, to balance out biases, offensive, or harmful content.
Source Data
The curators included online posts from the following sources sometime between 2014-2019:
- r/darkJokes, r/meanJokes, r/offensiveJokes
- Reddit microaggressions (Breitfeller et al., 2019)
- Toxic language detection Twitter corpora (Waseem & Hovy, 2016; Davidson et al., 2017; Founa et al., 2018)
- Data scraped from hate sites (Gab, Stormfront, r/incels, r/mensrights)
Initial Data Collection and Normalization
The curators wanted posts to be as self-contained as possible, therefore, they applied some filtering to prevent posts from being highly context-dependent. For Twitter data, they filtered out @-replies, retweets, and links, and subsample posts such that there is a smaller correlation between AAE and offensiveness (to avoid racial bias; Sap et al., 2019). For Reddit, Gab, and Stormfront, they only selected posts that were one sentence long, don't contain links, and are between 10 and 80 words. Furthemore, for Reddit, they automatically removed posts that target automated moderation.
Who are the source language producers?
Due to the nature of this corpus, there is no way to know who the speakers are. But, the speakers of the Reddit, Gab, and Stormfront posts are likely white men (see Gender by subreddit, Gab users, Stormfront description).
Annotations
Annotation process
For each post, Amazon Mechanical Turk workers indicate whether the post is offensive, whether the intent was to offend, and whether it contains lewd or sexual content. Only if annotators indicate potential offensiveness do they answer the group implication question. If the post targets or references a group or demographic, workers select or write which one(s); per selected group, they then write two to four stereotypes. Finally, workers are asked whether they think the speaker is part of one of the minority groups referenced by the post. The curators collected three annotations per post, and restricted the worker pool to the U.S. and Canada. The annotations in SBIC showed 82.4% pairwise agreement and Krippendorf’s α=0.45 on average.
Recent work has highlighted various negative side effects caused by annotating potentially abusive or harmful content (e.g., acute stress; Roberts, 2016). The curators mitigated these by limiting the number of posts that one worker could annotate in one day, paying workers above minimum wage ($7–12), and providing crisis management resources to the annotators.
Who are the annotators?
The annotators are Amazon Mechanical Turk workers aged 36±10 years old. The annotators consisted of 55% women, 42% men, and <1% non-binary and 82% identified as White, 4% Asian, 4% Hispanic, 4% Black. Information on their first language(s) and professional backgrounds was not collected.
Personal and Sensitive Information
Usernames are not included with the data, but the site where the post was collected is, so the user could potentially be recovered.
Considerations for Using the Data
Social Impact of Dataset
The curators recognize that studying Social Bias Frames necessarily requires confronting online content that may be offensive or disturbing but argue that deliberate avoidance does not eliminate such problems. By assessing social media content through the lens of Social Bias Frames, automatic flagging or AI-augmented writing interfaces may be analyzed for potentially harmful online content with detailed explanations for users or moderators to consider and verify. In addition, the collective analysis over large corpora can also be insightful for educating people on reducing unconscious biases in their language by encouraging empathy towards a targeted group.
Discussion of Biases
Because this is a corpus of social biases, a lot of posts contain implied or overt biases against the following groups (in decreasing order of prevalence):
- gender/sexuality
- race/ethnicity
- religion/culture
- social/political
- disability body/age
- victims
The curators warn that technology trained on this dataset could have side effects such as censorship and dialect-based racial bias.
Other Known Limitations
Because the curators found that the dataset is predominantly written in White-aligned English, they caution researchers to consider the potential for dialect or identity-based biases in labelling (Davidson et al.,2019; Sap et al., 2019a) before deploying technology based on SBIC.
Additional Information
Dataset Curators
This dataset was developed by Maarten Sap of the Paul G. Allen School of Computer Science & Engineering at the University of Washington, Saadia Gabriel, Lianhui Qin, Noah A Smith, and Yejin Choi of the Paul G. Allen School of Computer Science & Engineering and the Allen Institute for Artificial Intelligence, and Dan Jurafsky of the Linguistics & Computer Science Departments of Stanford University.
Licensing Information
The SBIC is licensed under the Creative Commons 4.0 License
Citation Information
@inproceedings{sap-etal-2020-social,
title = "Social Bias Frames: Reasoning about Social and Power Implications of Language",
author = "Sap, Maarten and
Gabriel, Saadia and
Qin, Lianhui and
Jurafsky, Dan and
Smith, Noah A. and
Choi, Yejin",
booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
month = jul,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.acl-main.486",
doi = "10.18653/v1/2020.acl-main.486",
pages = "5477--5490",
abstract = "Warning: this paper contains content that may be offensive or upsetting. Language has the power to reinforce stereotypes and project social biases onto others. At the core of the challenge is that it is rarely what is stated explicitly, but rather the implied meanings, that frame people{'}s judgments about others. For example, given a statement that {``}we shouldn{'}t lower our standards to hire more women,{''} most listeners will infer the implicature intended by the speaker - that {``}women (candidates) are less qualified.{''} Most semantic formalisms, to date, do not capture such pragmatic implications in which people express social biases and power differentials in language. We introduce Social Bias Frames, a new conceptual formalism that aims to model the pragmatic frames in which people project social biases and stereotypes onto others. In addition, we introduce the Social Bias Inference Corpus to support large-scale modelling and evaluation with 150k structured annotations of social media posts, covering over 34k implications about a thousand demographic groups. We then establish baseline approaches that learn to recover Social Bias Frames from unstructured text. We find that while state-of-the-art neural models are effective at high-level categorization of whether a given statement projects unwanted social bias (80{\%} F1), they are not effective at spelling out more detailed explanations in terms of Social Bias Frames. Our study motivates future work that combines structured pragmatic inference with commonsense reasoning on social implications.",
}
Contributions
Thanks to @thomwolf, @lewtun, @otakumesi, @mariamabarham, @lhoestq for adding this dataset.
- Downloads last month
- 1,988