(chen & manning) Tag: sourceedit |
(Mayberry & Miikkulainen1999) Tag: sourceedit |
||
| (9 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | This page monitor the line of research initialized by Chen and Manning (2014)<ref>Chen, D., & Manning, C. (2014). A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 740–750). Doha, Qatar: Association for Computational Linguistics.</ref> |
+ | This page monitor the line of research initialized by Chen and Manning (2014)<ref name=chen-manning14>Chen, D., & Manning, C. (2014). A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 740–750). Doha, Qatar: Association for Computational Linguistics.</ref> (although earlier attempt was made in a similar vein: Mayberry & Miikkulainen, 1999<ref>Marshall R. May- berry and Risto Miikkulainen. 1999. SARDSRN: A neural network shift-reduce parser. In Proc. IJCAI.</ref>). The common traits shared among the researches are the use of primitive features projected into vector spaces (not only words but also dependency label, parts-of-speech, etc.) and hidden layer(s) as a means of automatically combining given primitive features. |
| + | |||
| + | == Dependency parsing == |
||
| + | |||
| + | == Regularization == |
||
| + | |||
| + | Regulation is often mentioned in passing. Chen and Manning (2014)<ref name=chen-manning14/>, |
||
| + | Pei et al. (2015)<ref>Pei, W., Ge, T., & Chang, B. (2015). An effective neural network model for graph-based dependency parsing. In Proc. of ACL.</ref>, |
||
| + | Weiss et al. (2015)<ref>Weiss, D., Alberti, C., Collins, M., & Petrov, S. (2015). Structured Training for Neural Network Transition-Based Parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 323–333). Association for Computational Linguistics.</ref> all used L2 regularization. |
||
| + | |||
| + | == Combined with other tasks == |
||
| + | |||
| + | Trask et al. (2015)<ref>Andrew Trask & Phil Michalak & John Liu. 2015. SENSE2VEC - A FAST AND ACCURATE METHOD FOR WORD SENSE DISAMBIGUATION IN NEURAL WORD EMBEDDINGS.</ref> used combination of senses/POS tags and trained word sense disambiguation with dependency parsing. |
||
| + | |||
| + | == Multilingualism == |
||
| + | |||
| + | Guo et al. (2016)<ref>Guo, J., Che, W., Yarowsky, D., Wang, H., & Liu, T. (2016). A Representation Learning Framework for Multi-Source Transfer Parsing.</ref> use the neural network of Chen & Manning as the basis for their work. Smolentzov<ref>Smolentzov, A. LSTM dependency parser performance and Universal Dependencies.</ref> experimented with recurrent neural network. |
||
| + | |||
| + | TODO: [http://arxiv.org/pdf/1602.01595.pdf Ammar et al. (2016)] |
||
| + | |||
| + | == Speed == |
||
| + | |||
| + | One initial argument for neural dependency parsing was speed. For one, Chen & Manning (2014) said: |
||
| + | <blockquote>"Third, the use of many feature templates cause a less studied problem: in modern dependency parsers, most of the runtime is consumed not by the core parsing algorithm but in the feature extraction step (He et al., 2013). For instance, Bohnet (2010) reports that his baseline parser spends 99% of its time do- ing feature extraction, despite that being done in standard efficient ways."</blockquote> |
||
| + | |||
| + | However, later work (Rasooli & Tetreault, 2015)<ref>Mohammad Sadegh Rasooli and Joel Tetreault. Yara Parser: A Fast and Accurate Dependency Parser. arXiv:1503.06733v2 [cs.CL] Mar 2015.</ref> has demonstrated that one can extract [https://github.com/yahoo/YaraParser/blob/8b2cccded7756c57e46601945c33173b7d599baf/src/YaraParser/TransitionBasedSystem/Features/FeatureExtractor.java#L273 two-word] and even [https://github.com/yahoo/YaraParser/blob/8b2cccded7756c57e46601945c33173b7d599baf/src/YaraParser/TransitionBasedSystem/Features/FeatureExtractor.java#L306 three-word] features fast enough to enable a MaxEnt parser to run at the same or faster speed than a neural parser. |
||
== References == |
== References == |
||
| + | <references/> |
||
| − | <ref></ref> |
||
[[Category:Feed-forward neural network]] |
[[Category:Feed-forward neural network]] |
||
[[Category:Dependency parsing]] |
[[Category:Dependency parsing]] |
||
Latest revision as of 15:40, 3 May 2016
This page monitor the line of research initialized by Chen and Manning (2014)[1] (although earlier attempt was made in a similar vein: Mayberry & Miikkulainen, 1999[2]). The common traits shared among the researches are the use of primitive features projected into vector spaces (not only words but also dependency label, parts-of-speech, etc.) and hidden layer(s) as a means of automatically combining given primitive features.
Dependency parsing[]
Regularization[]
Regulation is often mentioned in passing. Chen and Manning (2014)[1], Pei et al. (2015)[3], Weiss et al. (2015)[4] all used L2 regularization.
Combined with other tasks[]
Trask et al. (2015)[5] used combination of senses/POS tags and trained word sense disambiguation with dependency parsing.
Multilingualism[]
Guo et al. (2016)[6] use the neural network of Chen & Manning as the basis for their work. Smolentzov[7] experimented with recurrent neural network.
TODO: Ammar et al. (2016)
Speed[]
One initial argument for neural dependency parsing was speed. For one, Chen & Manning (2014) said:
"Third, the use of many feature templates cause a less studied problem: in modern dependency parsers, most of the runtime is consumed not by the core parsing algorithm but in the feature extraction step (He et al., 2013). For instance, Bohnet (2010) reports that his baseline parser spends 99% of its time do- ing feature extraction, despite that being done in standard efficient ways."
However, later work (Rasooli & Tetreault, 2015)[8] has demonstrated that one can extract two-word and even three-word features fast enough to enable a MaxEnt parser to run at the same or faster speed than a neural parser.
References[]
- ↑ 1.0 1.1 Chen, D., & Manning, C. (2014). A Fast and Accurate Dependency Parser using Neural Networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 740–750). Doha, Qatar: Association for Computational Linguistics.
- ↑ Marshall R. May- berry and Risto Miikkulainen. 1999. SARDSRN: A neural network shift-reduce parser. In Proc. IJCAI.
- ↑ Pei, W., Ge, T., & Chang, B. (2015). An effective neural network model for graph-based dependency parsing. In Proc. of ACL.
- ↑ Weiss, D., Alberti, C., Collins, M., & Petrov, S. (2015). Structured Training for Neural Network Transition-Based Parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 323–333). Association for Computational Linguistics.
- ↑ Andrew Trask & Phil Michalak & John Liu. 2015. SENSE2VEC - A FAST AND ACCURATE METHOD FOR WORD SENSE DISAMBIGUATION IN NEURAL WORD EMBEDDINGS.
- ↑ Guo, J., Che, W., Yarowsky, D., Wang, H., & Liu, T. (2016). A Representation Learning Framework for Multi-Source Transfer Parsing.
- ↑ Smolentzov, A. LSTM dependency parser performance and Universal Dependencies.
- ↑ Mohammad Sadegh Rasooli and Joel Tetreault. Yara Parser: A Fast and Accurate Dependency Parser. arXiv:1503.06733v2 [cs.CL] Mar 2015.