Changes: Implicit semantic role labelling (State-of-the-art)

Revision as of 06:52, 11 July 2016

SemEval-2010 (FrameNet)

SemEval-2010 (Ruppenhofer et al., 2010)^[1] (Rel. = Relative, Abs. = Absolute, Over. = overlap measured by Dice coefficient)

	P	R	F1	NI recog.	Rel.	Abs.	P	R	F1	Over.	Notes	System ref	Performance ref
	SRL task			NI recog.	DNI vs. INI		DNI Linking				Notes	System ref	Performance ref
Shalmaneser	63.32	38.84	48.12	-	-	-	-	-	-	-			Ruppenhofer et al. (2010)^[1]
SEMAFOR 1.0	65.28	46.74	54.48	63.4	54.7	35	-	-	1.40	-		Chen et al. (2010)^[2]	Ruppenhofer et al. (2010)^[1], Silberer & Frank (2012)^[3]
CLR	67.02	11.21	19.21	-	-	-	-	-	-	-			Ruppenhofer et al. (2010)^[1]
GETARUNS++ /VENSES	-	-	-	- 8.0	64.2	5	-	-	1.21	-		Tonelli & Delmonte (2010)^[4]	Ruppenhofer et al. (2010)^[1], Silberer & Frank (2012)^[3]
Silberer & Frank	-	-	-	58	68	40	6.0	8.9	7.1	-	model M0	Silberer & Frank (2012)^[3]	Silberer & Frank (2012)^[3]
Laparra & Rigau (2012)	-	-	-	-	-	-	15	25	19	54		Laparra & Rigau (2012)^[5]	Laparra & Rigau (2012)^[5]
Laparra & Rigau (2013)	-	-	-	-	-	-	14	18	16	89	Table 4	Laparra & Rigau (2013)^[6]	Laparra & Rigau (2013)^[6]
Roth & Frank (2015)	-	-	-	-	-	-	21	8	12	-	Table 6 (p. 650)	Roth & Frank (2015)^[7]	Roth & Frank (2015)^[7]

SemEval-2010 (PropBank)

	P	R	F1	Over.	Notes	System ref	Performance ref
	DNI Linking				Notes	System ref	Performance ref
Feizabadi & Pado	10	20	13	-	in domain	Feizabadi & Pado (2015)^[8]	Feizabadi & Pado (2015)^[8]
Feizabadi & Pado	13	30	18	-	SEMEVAL train + BNB	Feizabadi & Pado (2015)^[8]	Feizabadi & Pado (2015)^[8]
Laparra and Rigau (2013)	12	16	14	-	-	Laparra and Rigau (2013)^[6]	Feizabadi & Pado (2015)^[8]

References

↑ ^1.0 ^1.1 ^1.2 ^1.3 ^1.4 Ruppenhofer, J., Sporleder, C., Morante, R., Baker, C., & Palmer, M. (2010). SemEval-2010 Task 10: Linking Events and Their Participants in Discourse. In Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010 (pp. 45–50). Uppsala, Sweden.
↑ Chen, D., Schneider, N., Das, D., & Smith, N. A. (2010). SEMAFOR: Frame argument resolution with log-linear models. In Proceedings of the 5th International Workshop on Semantic Evaluation (pp. 264–267). Uppsala, Sweden: Association for Computational Linguistics.
↑ ^3.0 ^3.1 ^3.2 ^3.3 Silberer, C., & Frank, A. (2012). Casting Implicit Role Linking As an Anaphora Resolution Task. In Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (pp. 1–10). Stroudsburg, PA, USA: Association for Computational Linguistics.
↑ Tonelli, S., & Delmonte, R. (2010). VENSES++: Adapting a deep semantic processing system to the identification of null instantiations. In Proceedings of the 5th International Workshop on Semantic Evaluations, ACL 2010 (pp. 296–299). Uppsala, Sweden: Association for Computational Linguistics.
↑ ^5.0 ^5.1 Laparra, E., & Rigau, G. (2012). Exploiting Explicit Annotations and Semantic Types for Implicit Argument Resolution. In Semantic Computing (ICSC), 2012 IEEE Sixth International Conference on (pp. 75–78). doi:10.1109/ICSC.2012.47
↑ ^6.0 ^6.1 ^6.2 Laparra, E., & Rigau, G. (2013). Sources of Evidence for Implicit Argument Resolution. In Proceedings of the 10th International Conference on Computational Semantics (IWCS2013). Potsdam, Germany.
↑ ^7.0 ^7.1 Roth, M., & Frank, A. (2015). Inducing Implicit Arguments from Comparable Texts: A Framework and its Applications. Computational Linguistics, 41(4), 625–664.
↑ ^8.0 ^8.1 ^8.2 Feizabadi, P. S., & Pado, S. (2015). Combining Seemingly Incompatible Corpora for Implicit Semantic Role Labeling. Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (* SEM 2015 ), 40–50.

[semeval-1] 1.0 ^1.1 ^1.2 ^1.3 ^1.4 Ruppenhofer, J., Sporleder, C., Morante, R., Baker, C., & Palmer, M. (2010). SemEval-2010 Task 10: Linking Events and Their Participants in Discourse. In Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010 (pp. 45–50). Uppsala, Sweden.

[2] Chen, D., Schneider, N., Das, D., & Smith, N. A. (2010). SEMAFOR: Frame argument resolution with log-linear models. In Proceedings of the 5th International Workshop on Semantic Evaluation (pp. 264–267). Uppsala, Sweden: Association for Computational Linguistics.

[silberer.frank.2012-3] 3.0 ^3.1 ^3.2 ^3.3 Silberer, C., & Frank, A. (2012). Casting Implicit Role Linking As an Anaphora Resolution Task. In Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (pp. 1–10). Stroudsburg, PA, USA: Association for Computational Linguistics.

[4] Tonelli, S., & Delmonte, R. (2010). VENSES++: Adapting a deep semantic processing system to the identification of null instantiations. In Proceedings of the 5th International Workshop on Semantic Evaluations, ACL 2010 (pp. 296–299). Uppsala, Sweden: Association for Computational Linguistics.

[Laparra.Rigau.2012)-5] 5.0 ^5.1 Laparra, E., & Rigau, G. (2012). Exploiting Explicit Annotations and Semantic Types for Implicit Argument Resolution. In Semantic Computing (ICSC), 2012 IEEE Sixth International Conference on (pp. 75–78). doi:10.1109/ICSC.2012.47

[Laparra.Rigau.2013-6] 6.0 ^6.1 ^6.2 Laparra, E., & Rigau, G. (2013). Sources of Evidence for Implicit Argument Resolution. In Proceedings of the 10th International Conference on Computational Semantics (IWCS2013). Potsdam, Germany.

[roth.frank.2015-7] 7.0 ^7.1 Roth, M., & Frank, A. (2015). Inducing Implicit Arguments from Comparable Texts: A Framework and its Applications. Computational Linguistics, 41(4), 625–664.

[Feizabadi.Pado.15-8] 8.0 ^8.1 ^8.2 Feizabadi, P. S., & Pado, S. (2015). Combining Seemingly Incompatible Corpora for Implicit Semantic Role Labeling. Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (* SEM 2015 ), 40–50.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

@@ Line 56: / Line 56: @@
 ! Roth & Frank (2015)
 | - || - || - || - || - || -
-| 0.21 || 0.08 || 0.12 || -
+| 21 || 8 || 12 || -
 | Table 6 (p. 650)
 | Roth & Frank (2015)<ref name=roth.frank.2015>Roth, M., & Frank, A. (2015). Inducing Implicit Arguments from Comparable Texts: A Framework and its Applications. Computational Linguistics, 41(4), 625–664.</ref>