Natural Language Understanding Wiki


Certainty Time Polarity FV
certain past/pres. positive factual
certain past/pres. negative counterfact.
non cert. any value any value non-fact.
any value future any value non-fact.
certain undersp. any value underspec.
undersp. past/pres. any value underspec.
undersp. undersp. any value underspec.
- - - no fact.
Table 1 (Minard et al. 2016): Possible combinations of factuality attributes.

From Minard et al. (2016)[1]:

  • "Certainty. Certainty relates to how sure the relevant source is about the mentioned event and admits the following three values: certain, non certain and underspecified.
  • Time. Time specifies the time when an event is reported to have taken place or is going to take place. Its values are past/present, future, and underspecified.
  • Polarity. Polarity captures whether an event is affirmed or negated and, consequently, it can be either positive or negative when there is not enough information available to detect the polarity of an event mention, its value is underspecified"
  • Factuality. "The combination of the certainty, time, and polarity attributes described above determines the factuality value (FV) of an event with respect to the relevant source. As shown in Table 1, the FV can assume five values: i.) factual; ii.) counterfactual; iii.) non-factual; iv.) underspecified; and v.) no factuality (no fact)."


  • MPQA annotated with factuality: van Son et al. (2014)[2]
  • FactBank: Saurí and Pustejovsky (2009)[3]
  • Fact-ItaBank (Minard et al., 2014)[4] and EVALITA 2016 Minard et al. (2016)[1]
  • French FactBank (Falk and Martin, 2017)[5]
  • Non-expert annotation: Lee et al. (2015)[6]

Other corpora that includes some aspects of factuality:

From Minard et al. (2016)[1]:

  • "the BioNLP 2009 Task 3 and BioNLP 2011 Shared Task aimed at recognizing if biomolecular events were affected by speculation or negation;
  • the CoNLL 2010 Share Task focused on hedge detection, i.e. identify speculated events, in biomedical texts;
  • the ACE Event Detection and Recognition tasks required systems to distinguish between asserted and non-asserted (e.g. hypothetical, desired, and promised) extracted events in news articles;
  • the 2012 *SEM Shared Task on Resolving The Scope of Negation focused one of its substasks on the identification of negated, i.e. counterfactual, events;
  • the Event Nugget Detection task at TAC KBP 2015 Event Track aimed at assessing the performance of systems in identifying events and their factual, or realis, value in news (Mitamura et al., 2015);
  • the 2015 and 2016 SemEval Clinical TempEval tasks required systems to assign the factuality value (i.e. attributed modality and polarity) to the extracted events in clinical notes."

See also[]


  1. 1.0 1.1 1.2 Minard, Anne-Lyse, Manuela Speranza, Tommaso Caselli, and Fondazione Bruno Kessler. "The EVALITA 2016 Event Factuality Annotation Task (FactA)." In Proceedings of Third Italian Conference on Computational Linguistics (CLiC-it 2016) & Fifth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2016). 2016.
  2. van Son, C., van Erp, M., Fokkens, A., & Vossen, P. (2014). Hope and Fear: Interpreting Perspectives by Integrating Sentiment and Event Factuality. In LREC 2014 (pp. 3857–3864).
  3. Saurí, R., & Pustejovsky, J. (2009). FactBank: a corpus annotated with event factuality. Language Resources and Evaluation, 43(3), 227.
  4. Anne-Lyse Minard, Alessandro Marchetti, and Manuela Speranza. 2014. Event Factuality in Italian: Annotation of News Stories from the ItaTimeBank. In Proceedings of CLiC-it 2014, First Italian Conference on Computational Linguistic.
  5. Falk, Ingrid, and Fabienne Martin. "Towards a lexicon of event-selecting predicates for a French FactBank." In Workshop Computational Semantics Beyond Events and Roses. 2017.
  6. Lee, K., Artzi, Y., Choi, Y., & Zettlemoyer, L. (2015). Event Detection and Factuality Assessment with Non-Expert Supervision. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1643–1648.