FrameNet-style semantic role labeling

Data

Use of lexicography (exemplar) corpus

Most work was done using both exemplar and fulltext annotations:

• Gildea & Jurafsky (2002): "Example sentences were chosen by searching the British National Corpus for instances of each target word. [...] Thus, the focus of the project was on completeness of examples for lexicographic needs, rather than on statistically representative data."
• Thompson et al. (2003): uses FrameNet 1.0 which doesn't have a fulltext corpus.
• Fleischman et al. (2003): "In each FrameNet sentence, a single target predicate is identified and all of its relevant frame elements are tagged with their semantic role"
• Senseval-3 uses FrameNet 1.1 including lexicographic data.
• Moschitti (2004): "For the FrameNet corpus (www.icsi.berkeley.edu/∼framenet) we extracted all 24,558 sen- tences from the 40 frames of Senseval 3 task (www.senseval.org) for the Automatic Labeling of Semantic Roles." (FrameNet 1.3 released in 2007 has only 1,700 sentences in the full-text portion)
• Giuglea & Moschitti (2006): "For the experiments on FN corpus, we extracted 58,384 sentences from the 319 frames that contain at least one verb annotation." (FrameNet 1.3 released in 2007 has only 1,700 sentences in the full-text portion)
• SemEval-2007 task 19 (Baker et al., 2007) did NOT prohibit the use of lexicographic part: "The major part of the training data for the task consisted of the current data release from FrameNet (Release 1.3), described in Sec.2 This was supplemented by additional training data made available through SemEval to participants in this task."
• Johansson & Nugues (2008): "We used the FrameNet example corpus and running-text corpus, from which we randomly sampled a training and test set."
• Matsubayashi et al. (2009): "We used the training set of the Semeval-2007 Shared task (Baker et al., 2007) in order to ascertain the contributions of role groups. This dataset consists of the corpus of FrameNet release 1.3 (containing roughly 150,000 annotations), and an additional full-text annotation dataset." (the fulltext part of FrameNet 1.3 has only 11,700 annotation sets)
• SemEval-2010 task 10 (Ruppenhofer et al. 2010) allows the use of "additional data, in particular the FrameNet and PropBank releases".
• Croce et al. (2011): "We used the FrameNet version 1.3 with the 90/10% split between training and test set (i.e 271,560 and 30,173 examples respectively), as defined in (Johansson and Nugues, 2008b)"
• Laparra & Rigau (2012): "The dataset also includes the annotation files for the lexical units and the full-text annotated corpus from FrameNet."

Some don't make use of lexicographic annotations:

• Das et al. (2010): "We found that using exemplar sentences directly to train our models hurt performance as evaluated on SemEval’07 data, even though the number of exemplar sentences is an order of magnitude larger than the number of sentences in our training set (§2.2). This is presumably because the exemplars are neither representative as a sample nor similar to the test data. Instead, we make use of these exemplars in features (§4.2)."
• Roth & Lapata (2015): "Following previous work on FrameNet-based SRL, we use the full text annotation data set, which contains 23,087 frame instances."
• FitzGerald et al. (2015): "For frame-semantic parsing using FrameNet conventions (Baker et al., 1998), we follow Das et al. (2014) and Hermann et al. (2014) in using the full-text annotations of the FrameNet 1.5 release and follow their data splits."

Irrelevant for this classification: Shi & Mihalcea (2004) (rule-based)

Not clear: Erk & Pado (2006)

Dealing with unseen lexical units

From Palmer et al. (2001):

"For example, Das et al. (2010) introduce a latent variable ranging over seen targets, allowing them to infer likely frames for unseen words, and the SRL system of Johansson and Nugues (2007) uses WordNet to generalise to unseen lemmas. In a similar vein, Burchardt et al. (2005) propose a system that generalizes over WordNet synsets to guess frames for unknown words. Pennacchiotti et al. (2008) compare WordNet-based and distributional approaches to inferring frames and conclude that a combination of the two leads to the best results, while (Cao et al., 2008) discuss how different distributional models can be utilised. Several approaches have also addressed other coverage problems, e.g., how to automatically expand the number of example sentences for a given lexical unit (Pado ́et al., 2008; Furstenau and Lapata, 2009). Another related approach is that of generalizing over semantic roles. Baldewein et al. (2004) use the FrameNet hierarchy to model the similarity of roles, boosting seldom-seen instances by reusing training data for similar roles, though without significant gains in performance. The most extensive study on role generalization to date (Matsubayashi et al., 2009) compares different ways of grouping roles—exploiting hierarchical relations in FrameNet, generalizing via role names, utilising role types, and using thematic roles from VerbNet—with the best results from using all groups together"