Narrative cloze is a task proposed by Chambers and Jurafsky (2008) It is widely used to evaluate models of script knowledge (Pichotta & Mooney, 2016a; Pichotta & Mooney, 2016b; Jans et al., 2012; Rudinger et al., 2015a; Rudinger et al. (2015b))
From Pichotta & Mooney (2016b): "The exact definition of the Narrative Cloze evaluation depends on the formulation of events used in a script system. For example, Cham- bers and Jurafsky (2008), Jans et al. (2012), and Rudinger et al. (2015) evaluate inference of held- out (verb, dependency) pairs from documents; Pi- chotta and Mooney (2014) evaluate inference of verbs with coreference information about multi- ple arguments; and Pichotta and Mooney (2016) evaluate inference of verbs with noun informa- tion about multiple arguments. In order to gather human judgments of inference quality, the latter also learn an encoder-decoder LSTM network for transforming verbs and noun arguments into En- glish text to present to annotators for evaluation."
- ↑ Chambers, N., and Jurafsky, D. 2008. Unsupervised learning of narrative event chains. In Proceedings of ACL, 789– 797. Association for Computational Linguistics.
- ↑ Pichotta, K., & Mooney, R. J. (2016a). Learning Statistical Scripts With LSTM Recurrent Neural Networks. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI-16), Phoenix, Arizona.
- ↑ Pichotta, K., & Mooney, R. J. (2016b). Using Sentence-Level LSTM Language Models for Script Inference. Retrieved from http://arxiv.org/abs/1604.02993
- ↑ Bram Jans, Steven Bethard, Ivan Vulic, and Marie Francine Moens. 2012. Skip n- grams and ranking functions for predicting script events. In Proceedings of the 13th Conference of the European Chapter of the Association for Com- putational Linguistics (EACL-12), pages 336–344.
- ↑ Rachel Rudinger, Pushpendre Rastogi, Francis Ferraro, and Benjamin Van Durme. 2015a. Script induction as language modeling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP-15).
- ↑ Rudinger, Rachel, Vera Demberg, Ashutosh Modi, Benjamin Van Durme, and Manfred Pinkal. 2015b. "Learning to predict script events from domain-specific text." Lexical and Computational Semantics (* SEM 2015): 205.