(auto_conll and gold_conll) Tags: Visual edit apiedit |
(link) Tag: Visual edit |
||
| (4 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | [[File:Inter-annotator-agreement-conll-2012.png|thumb|220x220px|Inter-annotator agreement in CoNLL-2012]] |
||
| ⚫ | The data can be downloaded from [http://conll.cemantix.org/2011/data.html |
||
| + | Main references: |
||
| + | * 2011: Pradhan et al. (2011)<ref>Pradhan, S., Ramshaw, L., Marcus, M., Palmer, M., Weischedel, R., & Xue, N. (2011). CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In ''Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task'' (pp. 1–27). Association for Computational Linguistics.</ref> |
||
| + | * 2012: Pradhan et al. (2012)<ref>Pradhan, S., Moschitti, A., Xue, N., Uryupina, O., Zhang, Y.: CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In: Joint Con- ference on EMNLP and CoNLL-Shared Task, pp. 1–40. Association for Computa- tional Linguistics (2012)</ref> |
||
| ⚫ | The data can be downloaded from [http://conll.cemantix.org/2011/data.html (2011)] and [http://conll.cemantix.org/2012/data.html (2012)].<blockquote>'''Notice! The data is not ready after downloaded. Follow the instructions to create *.v2_auto_conll and *.v2_gold_conll files.'''</blockquote>Difference between .*_auto_conll and .*_gold_conll: from [https://github.com/stanfordnlp/CoreNLP/issues/62#issuecomment-77440856 Xiao Cheng]:<blockquote>auto_conll uses parser's parse tree and gold_conll uses human annotated parse tree. Both have the same gold mentions ( thus auto_conll might have some gold mention not being a NP due to parser error )</blockquote>About singletons: from Durrett and Klein (2013)<ref>Durrett, G., & Klein, D. (2013). Easy victories and uphill battles in coreference resolution. ''EMNLP ’13'', (October), 1971–1982.</ref>: "Singletons are always removed before evaluation because the OntoNotes corpus does not annotate them" (CoNLL-2011 and 2012 are part of OntoNotes). |
||
| + | |||
| + | == References == |
||
| + | <references />[[Category:Datasets]] |
||
[[Category:Coreference resolution]] |
[[Category:Coreference resolution]] |
||
Latest revision as of 09:00, 26 October 2017
Inter-annotator agreement in CoNLL-2012
Main references:
The data can be downloaded from (2011) and (2012).
Notice! The data is not ready after downloaded. Follow the instructions to create *.v2_auto_conll and *.v2_gold_conll files.
Difference between .*_auto_conll and .*_gold_conll: from Xiao Cheng:
auto_conll uses parser's parse tree and gold_conll uses human annotated parse tree. Both have the same gold mentions ( thus auto_conll might have some gold mention not being a NP due to parser error )
About singletons: from Durrett and Klein (2013)[3]: "Singletons are always removed before evaluation because the OntoNotes corpus does not annotate them" (CoNLL-2011 and 2012 are part of OntoNotes).
References[]
- ↑ Pradhan, S., Ramshaw, L., Marcus, M., Palmer, M., Weischedel, R., & Xue, N. (2011). CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task (pp. 1–27). Association for Computational Linguistics.
- ↑ Pradhan, S., Moschitti, A., Xue, N., Uryupina, O., Zhang, Y.: CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes. In: Joint Con- ference on EMNLP and CoNLL-Shared Task, pp. 1–40. Association for Computa- tional Linguistics (2012)
- ↑ Durrett, G., & Klein, D. (2013). Easy victories and uphill battles in coreference resolution. EMNLP ’13, (October), 1971–1982.