Hugging Face's logo

Datasets:
super_glue

Tasks:
Text Classification
Token Classification
Question Answering
Sub-tasks:
natural-language-inference
word-sense-disambiguation
coreference-resolution
Languages:
English
Multilinguality:
monolingual
Size Categories:
10K<n<100K
Language Creators:
other
Annotations Creators:
expert-generated
Source Datasets:
extended|other
ArXiv:
Tags:
superglue
NLU
natural language understanding
License:
Dataset card Viewer Files Community
14
Dataset Viewer
Auto-converted to Parquet
Viewer
sentence1
stringlengths
11
296
sentence2
stringlengths
11
296
idx
int32
0
1.1k
label
class label
2 classes
The cat sat on the mat.
The cat did not sit on the mat.
0
1not_entailment
The cat did not sit on the mat.
The cat sat on the mat.
1
1not_entailment
When you've got no snow, it's really hard to learn a snow sport so we looked at all the different ways I could mimic being on snow without actually being on snow.
When you've got snow, it's really hard to learn a snow sport so we looked at all the different ways I could mimic being on snow without actually being on snow.
2
1not_entailment
When you've got snow, it's really hard to learn a snow sport so we looked at all the different ways I could mimic being on snow without actually being on snow.
When you've got no snow, it's really hard to learn a snow sport so we looked at all the different ways I could mimic being on snow without actually being on snow.
3
1not_entailment
Out of the box, Ouya supports media apps such as Twitch.tv and XBMC media player.
Out of the box, Ouya doesn't support media apps such as Twitch.tv and XBMC media player.
4
1not_entailment
Out of the box, Ouya doesn't support media apps such as Twitch.tv and XBMC media player.
Out of the box, Ouya supports media apps such as Twitch.tv and XBMC media player.
5
1not_entailment
Out of the box, Ouya supports media apps such as Twitch.tv and XBMC media player.
Out of the box, Ouya supports Twitch.tv and XBMC media player.
6
0entailment
Out of the box, Ouya supports Twitch.tv and XBMC media player.
Out of the box, Ouya supports media apps such as Twitch.tv and XBMC media player.
7
0entailment
Considering this definition, it is surprising to find frequent use of sarcastic language in opinionated user generated content.
Considering this definition, it is not surprising to find frequent use of sarcastic language in opinionated user generated content.
8
1not_entailment
Considering this definition, it is not surprising to find frequent use of sarcastic language in opinionated user generated content.
Considering this definition, it is surprising to find frequent use of sarcastic language in opinionated user generated content.
9
1not_entailment
The new gaming console is affordable.
The new gaming console is unaffordable.
10
1not_entailment
The new gaming console is unaffordable.
The new gaming console is affordable.
11
1not_entailment
Brexit is an irreversible decision, Sir Mike Rake, the chairman of WorldPay and ex-chairman of BT group, said as calls for a second EU referendum were sparked last week.
Brexit is a reversible decision, Sir Mike Rake, the chairman of WorldPay and ex-chairman of BT group, said as calls for a second EU referendum were sparked last week.
12
1not_entailment
Brexit is a reversible decision, Sir Mike Rake, the chairman of WorldPay and ex-chairman of BT group, said as calls for a second EU referendum were sparked last week.
Brexit is an irreversible decision, Sir Mike Rake, the chairman of WorldPay and ex-chairman of BT group, said as calls for a second EU referendum were sparked last week.
13
1not_entailment
We built our society on unclean energy.
We built our society on clean energy.
14
1not_entailment
We built our society on clean energy.
We built our society on unclean energy.
15
1not_entailment
Pursuing a strategy of nonviolent protest, Gandhi took the administration by surprise and won concessions from the authorities.
Pursuing a strategy of violent protest, Gandhi took the administration by surprise and won concessions from the authorities.
16
1not_entailment
Pursuing a strategy of violent protest, Gandhi took the administration by surprise and won concessions from the authorities.
Pursuing a strategy of nonviolent protest, Gandhi took the administration by surprise and won concessions from the authorities.
17
1not_entailment
Pursuing a strategy of nonviolent protest, Gandhi took the administration by surprise and won concessions from the authorities.
Pursuing a strategy of protest, Gandhi took the administration by surprise and won concessions from the authorities.
18
0entailment
Pursuing a strategy of protest, Gandhi took the administration by surprise and won concessions from the authorities.
Pursuing a strategy of nonviolent protest, Gandhi took the administration by surprise and won concessions from the authorities.
19
1not_entailment
And if both apply, they are essentially impossible.
And if both apply, they are essentially possible.
20
1not_entailment
And if both apply, they are essentially possible.
And if both apply, they are essentially impossible.
21
1not_entailment
Writing Java is not too different from programming with handcuffs.
Writing Java is similar to programming with handcuffs.
22
0entailment
Writing Java is similar to programming with handcuffs.
Writing Java is not too different from programming with handcuffs.
23
0entailment
The market is about to get harder, but not impossible to navigate.
The market is about to get harder, but possible to navigate.
24
0entailment
The market is about to get harder, but possible to navigate.
The market is about to get harder, but not impossible to navigate.
25
0entailment
Even after now finding out that it's animal feed, I won't ever stop being addicted to Flamin' Hot Cheetos.
Even after now finding out that it's animal feed, I will never stop being addicted to Flamin' Hot Cheetos.
26
0entailment
Even after now finding out that it's animal feed, I will never stop being addicted to Flamin' Hot Cheetos.
Even after now finding out that it's animal feed, I won't ever stop being addicted to Flamin' Hot Cheetos.
27
0entailment
He did not disagree with the party's position, but felt that if he resigned, his popularity with Indians would cease to stifle the party's membership.
He agreed with the party's position, but felt that if he resigned, his popularity with Indians would cease to stifle the party's membership.
28
0entailment
He agreed with the party's position, but felt that if he resigned, his popularity with Indians would cease to stifle the party's membership.
He did not disagree with the party's position, but felt that if he resigned, his popularity with Indians would cease to stifle the party's membership.
29
0entailment
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would be expected.
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would not be unexpected.
30
0entailment
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would not be unexpected.
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would be expected.
31
0entailment
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would be expected.
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, it would be expected to negatively impact the pipeline results.
32
0entailment
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, it would be expected to negatively impact the pipeline results.
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would be expected.
33
0entailment
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would be expected.
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, it would not be unexpected for it to negatively impact the pipeline results.
34
0entailment
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, it would not be unexpected for it to negatively impact the pipeline results.
If the pipeline tokenization scheme does not correspond to the one that was used when a model was created, a negative impact on the pipeline results would be expected.
35
0entailment
The water is too hot.
The water is too cold.
36
1not_entailment
The water is too cold.
The water is too hot.
37
1not_entailment
Falcon Heavy is the largest rocket since NASA's Saturn V booster, which was used for the Moon missions in the 1970s.
Falcon Heavy is the smallest rocket since NASA's Saturn V booster, which was used for the Moon missions in the 1970s.
38
1not_entailment
Falcon Heavy is the smallest rocket since NASA's Saturn V booster, which was used for the Moon missions in the 1970s.
Falcon Heavy is the largest rocket since NASA's Saturn V booster, which was used for the Moon missions in the 1970s.
39
1not_entailment
Adenoiditis symptoms often persist for ten or more days, and often include pus-like discharge from nose.
Adenoiditis symptoms often pass within ten days or less, and often include pus-like discharge from nose.
40
1not_entailment
Adenoiditis symptoms often pass within ten days or less, and often include pus-like discharge from nose.
Adenoiditis symptoms often persist for ten or more days, and often include pus-like discharge from nose.
41
1not_entailment
In example (1) it is quite straightforward to see the exaggerated positive sentiment used in order to convey strong negative feelings.
In example (1) it is quite difficult to see the exaggerated positive sentiment used in order to convey strong negative feelings.
42
1not_entailment
In example (1) it is quite difficult to see the exaggerated positive sentiment used in order to convey strong negative feelings.
In example (1) it is quite straightforward to see the exaggerated positive sentiment used in order to convey strong negative feelings.
43
1not_entailment
In example (1) it is quite straightforward to see the exaggerated positive sentiment used in order to convey strong negative feelings.
In example (1) it is quite easy to see the exaggerated positive sentiment used in order to convey strong negative feelings.
44
0entailment
In example (1) it is quite easy to see the exaggerated positive sentiment used in order to convey strong negative feelings.
In example (1) it is quite straightforward to see the exaggerated positive sentiment used in order to convey strong negative feelings.
45
0entailment
In example (1) it is quite straightforward to see the exaggerated positive sentiment used in order to convey strong negative feelings.
In example (1) it is quite important to see the exaggerated positive sentiment used in order to convey strong negative feelings.
46
1not_entailment
In example (1) it is quite important to see the exaggerated positive sentiment used in order to convey strong negative feelings.
In example (1) it is quite straightforward to see the exaggerated positive sentiment used in order to convey strong negative feelings.
47
1not_entailment
Some dogs like to scratch their ears.
Some animals like to scratch their ears.
48
0entailment
Some animals like to scratch their ears.
Some dogs like to scratch their ears.
49
1not_entailment
Cruz has frequently derided as "amnesty" various plans that confer legal status or citizenship on people living in the country illegally.
Cruz has frequently derided as "amnesty" various bills that confer legal status or citizenship on people living in the country illegally.
50
1not_entailment
Cruz has frequently derided as "amnesty" various bills that confer legal status or citizenship on people living in the country illegally.
Cruz has frequently derided as "amnesty" various plans that confer legal status or citizenship on people living in the country illegally.
51
0entailment
Most of the graduates of my program have moved on to other things because the jobs suck.
Some of the graduates of my program have moved on to other things because the jobs suck.
52
0entailment
Some of the graduates of my program have moved on to other things because the jobs suck.
Most of the graduates of my program have moved on to other things because the jobs suck.
53
1not_entailment
In many developed areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
In many areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
54
0entailment
In many areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
In many developed areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
55
1not_entailment
We consider some context words as positive examples and sample negatives at random from the dictionary.
We consider some words as positive examples and sample negatives at random from the dictionary.
56
0entailment
We consider some words as positive examples and sample negatives at random from the dictionary.
We consider some context words as positive examples and sample negatives at random from the dictionary.
57
1not_entailment
We consider some context words as positive examples and sample negatives at random from the dictionary.
We consider all context words as positive examples and sample many negatives at random from the dictionary.
58
1not_entailment
We consider all context words as positive examples and sample many negatives at random from the dictionary.
We consider some context words as positive examples and sample negatives at random from the dictionary.
59
1not_entailment
We consider some context words as positive examples and sample negatives at random from the dictionary.
We consider many context words as positive examples and sample negatives at random from the dictionary.
60
1not_entailment
We consider many context words as positive examples and sample negatives at random from the dictionary.
We consider some context words as positive examples and sample negatives at random from the dictionary.
61
0entailment
We consider all context words as positive examples and sample negatives at random from the dictionary.
We consider all words as positive examples and sample negatives at random from the dictionary.
62
1not_entailment
We consider all words as positive examples and sample negatives at random from the dictionary.
We consider all context words as positive examples and sample negatives at random from the dictionary.
63
0entailment
All dogs like to scratch their ears.
All animals like to scratch their ears.
64
1not_entailment
All animals like to scratch their ears.
All dogs like to scratch their ears.
65
0entailment
Cruz has frequently derided as "amnesty" any plan that confers legal status or citizenship on people living in the country illegally.
Cruz has frequently derided as "amnesty" any bill that confers legal status or citizenship on people living in the country illegally.
66
0entailment
Cruz has frequently derided as "amnesty" any bill that confers legal status or citizenship on people living in the country illegally.
Cruz has frequently derided as "amnesty" any plan that confers legal status or citizenship on people living in the country illegally.
67
1not_entailment
Most of the graduates of my program have moved on to other things because the jobs suck.
None of the graduates of my program have moved on to other things because the jobs suck.
68
1not_entailment
None of the graduates of my program have moved on to other things because the jobs suck.
Most of the graduates of my program have moved on to other things because the jobs suck.
69
1not_entailment
Most of the graduates of my program have moved on to other things because the jobs suck.
All of the graduates of my program have moved on to other things because the jobs suck.
70
1not_entailment
All of the graduates of my program have moved on to other things because the jobs suck.
Most of the graduates of my program have moved on to other things because the jobs suck.
71
1not_entailment
In all areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
In all developed areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
72
0entailment
In all developed areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
In all areas, human activity has changed the form of river channels, altering magnitudes and frequencies of flooding.
73
1not_entailment
Tom and Adam were whispering in the theater.
Tom and Adam were whispering quietly in the theater.
74
0entailment
Tom and Adam were whispering quietly in the theater.
Tom and Adam were whispering in the theater.
75
0entailment
Tom and Adam were whispering in the theater.
Tom and Adam were whispering loudly in the theater.
76
1not_entailment
Tom and Adam were whispering loudly in the theater.
Tom and Adam were whispering in the theater.
77
0entailment
Prior to the dance, which is voluntary, students are told to fill out a card by selecting five people they want to dance with.
Prior to the dance, which is voluntary, students are told to fill out a card by selecting five different people they want to dance with.
78
0entailment
Prior to the dance, which is voluntary, students are told to fill out a card by selecting five different people they want to dance with.
Prior to the dance, which is voluntary, students are told to fill out a card by selecting five people they want to dance with.
79
0entailment
Notifications about Farmville and other crap had become unbearable, then the shift to the non-chronological timeline happened and the content from your friends started to be replaced by ads and other cringy wannabe-viral campaigns.
Notifications about Farmville and other crappy apps had become unbearable, then the shift to the non-chronological timeline happened and the content from your friends started to be replaced by ads and other cringy wannabe-viral campaigns.
80
0entailment
Notifications about Farmville and other crappy apps had become unbearable, then the shift to the non-chronological timeline happened and the content from your friends started to be replaced by ads and other cringy wannabe-viral campaigns.
Notifications about Farmville and other crap had become unbearable, then the shift to the non-chronological timeline happened and the content from your friends started to be replaced by ads and other cringy wannabe-viral campaigns.
81
0entailment
Chicago City Hall is the official seat of government of the City of Chicago.
Chicago City Hall is the official seat of government of Chicago.
82
0entailment
Chicago City Hall is the official seat of government of Chicago.
Chicago City Hall is the official seat of government of the City of Chicago.
83
0entailment
The question generation aspect is unique to our formulation, and corresponds roughly to identifying what semantic role labels are present in previous formulations of the task.
The question generation aspect is unique to our formulation, and corresponds roughly to identifying what semantic role labels are present in previous other formulations of the task.
84
0entailment
The question generation aspect is unique to our formulation, and corresponds roughly to identifying what semantic role labels are present in previous other formulations of the task.
The question generation aspect is unique to our formulation, and corresponds roughly to identifying what semantic role labels are present in previous formulations of the task.
85
0entailment
John ate pasta for dinner.
John ate pasta for supper.
86
0entailment
John ate pasta for supper.
John ate pasta for dinner.
87
0entailment
John ate pasta for dinner.
John ate pasta for breakfast.
88
1not_entailment
John ate pasta for breakfast.
John ate pasta for dinner.
89
1not_entailment
House Speaker Paul Ryan was facing problems from fellow Republicans dissatisfied with his leadership.
House Speaker Paul Ryan was facing problems from fellow Republicans unhappy with his leadership.
90
0entailment
House Speaker Paul Ryan was facing problems from fellow Republicans unhappy with his leadership.
House Speaker Paul Ryan was facing problems from fellow Republicans dissatisfied with his leadership.
91
0entailment
House Speaker Paul Ryan was facing problems uniquely from fellow Republicans dissatisfied with his leadership.
House Speaker Paul Ryan was facing problems uniquely from fellow Republicans supportive of his leadership.
92
1not_entailment
House Speaker Paul Ryan was facing problems uniquely from fellow Republicans supportive of his leadership.
House Speaker Paul Ryan was facing problems uniquely from fellow Republicans dissatisfied with his leadership.
93
1not_entailment
I can actually see him climbing into a Lincoln saying this.
I can actually see him getting into a Lincoln saying this.
94
0entailment
I can actually see him getting into a Lincoln saying this.
I can actually see him climbing into a Lincoln saying this.
95
0entailment
I can actually see him climbing into a Lincoln saying this.
I can actually see him climbing into a Mazda saying this.
96
1not_entailment
I can actually see him climbing into a Mazda saying this.
I can actually see him climbing into a Lincoln saying this.
97
1not_entailment
The villain is the character who tends to have a negative effect on other characters.
The villain is the character who tends to have a negative impact on other characters.
98
0entailment
The villain is the character who tends to have a negative impact on other characters.
The villain is the character who tends to have a negative effect on other characters.
99
0entailment
End of preview. Expand in Dataset Viewer.

Dataset Card for "super_glue"

Dataset Summary

SuperGLUE (https://super.gluebenchmark.com/) is a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a new public leaderboard.

Supported Tasks and Leaderboards

More Information Needed

Languages

More Information Needed

Dataset Structure

Data Instances

axb

  • Size of downloaded dataset files: 0.03 MB
  • Size of the generated dataset: 0.24 MB
  • Total amount of disk used: 0.27 MB

An example of 'test' looks as follows.

axg

  • Size of downloaded dataset files: 0.01 MB
  • Size of the generated dataset: 0.05 MB
  • Total amount of disk used: 0.06 MB

An example of 'test' looks as follows.

boolq

  • Size of downloaded dataset files: 4.12 MB
  • Size of the generated dataset: 10.40 MB
  • Total amount of disk used: 14.52 MB

An example of 'train' looks as follows.

cb

  • Size of downloaded dataset files: 0.07 MB
  • Size of the generated dataset: 0.20 MB
  • Total amount of disk used: 0.28 MB

An example of 'train' looks as follows.

copa

  • Size of downloaded dataset files: 0.04 MB
  • Size of the generated dataset: 0.13 MB
  • Total amount of disk used: 0.17 MB

An example of 'train' looks as follows.

Data Fields

The data fields are the same among all splits.

axb

  • sentence1: a string feature.
  • sentence2: a string feature.
  • idx: a int32 feature.
  • label: a classification label, with possible values including entailment (0), not_entailment (1).

axg

  • premise: a string feature.
  • hypothesis: a string feature.
  • idx: a int32 feature.
  • label: a classification label, with possible values including entailment (0), not_entailment (1).

boolq

  • question: a string feature.
  • passage: a string feature.
  • idx: a int32 feature.
  • label: a classification label, with possible values including False (0), True (1).

cb

  • premise: a string feature.
  • hypothesis: a string feature.
  • idx: a int32 feature.
  • label: a classification label, with possible values including entailment (0), contradiction (1), neutral (2).

copa

  • premise: a string feature.
  • choice1: a string feature.
  • choice2: a string feature.
  • question: a string feature.
  • idx: a int32 feature.
  • label: a classification label, with possible values including choice1 (0), choice2 (1).

Data Splits

axb

test
axb 1104

axg

test
axg 356

boolq

train validation test
boolq 9427 3270 3245

cb

train validation test
cb 250 56 250

copa

train validation test
copa 400 100 500

Dataset Creation

Curation Rationale

More Information Needed

Source Data

Initial Data Collection and Normalization

More Information Needed

Who are the source language producers?

More Information Needed

Annotations

Annotation process

More Information Needed

Who are the annotators?

More Information Needed

Personal and Sensitive Information

More Information Needed

Considerations for Using the Data

Social Impact of Dataset

More Information Needed

Discussion of Biases

More Information Needed

Other Known Limitations

More Information Needed

Additional Information

Dataset Curators

More Information Needed

Licensing Information

The primary SuperGLUE tasks are built on and derived from existing datasets. We refer users to the original licenses accompanying each dataset, but it is our understanding that these licenses allow for their use and redistribution in a research context.

Citation Information

If you use SuperGLUE, please cite all the datasets you use in any papers that come out of your work. In addition, we encourage you to use the following BibTeX citation for SuperGLUE itself:

@article{wang2019superglue,
  title={Super{GLUE}: A Stickier Benchmark for General-Purpose Language Understanding Systems},
  author={Alex Wang and Yada Pruksachatkun and Nikita Nangia and Amanpreet Singh and Julian Michael and Felix Hill and Omer Levy and Samuel R. Bowman},
  journal={arXiv preprint 1905.00537},
  year={2019}
}
@inproceedings{clark2019boolq,
  title={{B}ool{Q}: Exploring the Surprising Difficulty of Natural Yes/No Questions},
  author={Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina},
  booktitle={Proceedings of NAACL-HLT 2019},
  year={2019}
}
@inproceedings{demarneffe:cb,
  title={{The CommitmentBank}: Investigating projection in naturally occurring discourse},
  author={De Marneffe, Marie-Catherine and Simons, Mandy and Tonhauser, Judith},
  note={To appear in proceedings of Sinn und Bedeutung 23. Data can be found at https://github.com/mcdm/CommitmentBank/},
  year={2019}
}
@inproceedings{roemmele2011choice,
  title={Choice of plausible alternatives: An evaluation of commonsense causal reasoning},
  author={Roemmele, Melissa and Bejan, Cosmin Adrian and Gordon, Andrew S.},
  booktitle={2011 AAAI Spring Symposium Series},
  year={2011}
}
@inproceedings{khashabi2018looking,
  title={Looking beyond the surface: A challenge set for reading comprehension over multiple sentences},
  author={Khashabi, Daniel and Chaturvedi, Snigdha and Roth, Michael and Upadhyay, Shyam and Roth, Dan},
  booktitle={Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  pages={252--262},
  year={2018}
}
@article{zhang2018record,
  title={{ReCoRD}: Bridging the Gap between Human and Machine Commonsense Reading Comprehension},
  author={Sheng Zhang and Xiaodong Liu and Jingjing Liu and Jianfeng Gao and Kevin Duh and Benjamin Van Durme},
  journal={arXiv preprint 1810.12885},
  year={2018}
}
@incollection{dagan2006pascal,
  title={The {PASCAL} recognising textual entailment challenge},
  author={Dagan, Ido and Glickman, Oren and Magnini, Bernardo},
  booktitle={Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment},
  pages={177--190},
  year={2006},
  publisher={Springer}
}
@article{bar2006second,
  title={The second {PASCAL} recognising textual entailment challenge},
  author={Bar Haim, Roy and Dagan, Ido and Dolan, Bill and Ferro, Lisa and Giampiccolo, Danilo and Magnini, Bernardo and Szpektor, Idan},
  year={2006}
}
@inproceedings{giampiccolo2007third,
  title={The third {PASCAL} recognizing textual entailment challenge},
  author={Giampiccolo, Danilo and Magnini, Bernardo and Dagan, Ido and Dolan, Bill},
  booktitle={Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing},
  pages={1--9},
  year={2007},
  organization={Association for Computational Linguistics},
}
@article{bentivogli2009fifth,
  title={The Fifth {PASCAL} Recognizing Textual Entailment Challenge},
  author={Bentivogli, Luisa and Dagan, Ido and Dang, Hoa Trang and Giampiccolo, Danilo and Magnini, Bernardo},
  booktitle={TAC},
  year={2009}
}
@inproceedings{pilehvar2018wic,
  title={{WiC}: The Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations},
  author={Pilehvar, Mohammad Taher and Camacho-Collados, Jose},
  booktitle={Proceedings of NAACL-HLT},
  year={2019}
}
@inproceedings{rudinger2018winogender,
  title={Gender Bias in Coreference Resolution},
  author={Rudinger, Rachel  and  Naradowsky, Jason  and  Leonard, Brian  and  {Van Durme}, Benjamin},
  booktitle={Proceedings of NAACL-HLT},
  year={2018}
}
@inproceedings{poliak2018dnc,
  title={Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation},
  author={Poliak, Adam and Haldar, Aparajita and Rudinger, Rachel and Hu, J. Edward and Pavlick, Ellie and White, Aaron Steven and {Van Durme}, Benjamin},
  booktitle={Proceedings of EMNLP},
  year={2018}
}
@inproceedings{levesque2011winograd,
  title={The {W}inograd schema challenge},
  author={Levesque, Hector J and Davis, Ernest and Morgenstern, Leora},
  booktitle={{AAAI} Spring Symposium: Logical Formalizations of Commonsense Reasoning},
  volume={46},
  pages={47},
  year={2011}
}

Contributions

Thanks to @thomwolf, @lewtun, @patrickvonplaten for adding this dataset.

Downloads last month
169,043
Edit dataset card Papers with Code
Homepage:
super.gluebenchmark.com
Paper:
arxiv.org
Size of downloaded dataset files:
58.4 MB
Size of the auto-converted Parquet files:
103 MB
Number of rows:
196,313

Models trained or fine-tuned on super_glue

Browse 273 models trained on this dataset