Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing

Brian Roark; Asaf Bachrach; Carlos Cardenas; Christophe Pallier

doi:10.3115/1699510.1699553

Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing

Brian Roark
, Asaf Bachrach
, Carlos Cardenas
, Christophe Pallier

Center for Spoken Language Understanding
Oregon Health & Science University
Service Hospitalier Frédéric Joliot
Massachusetts Institute of Technology

Research output: Contribution to conference › Paper › peer-review

155 Citations (Scopus)

Abstract

A number of recent publications have made use of the incremental output of stochastic parsers to derive measures of high utility for psycholinguistic modeling, following the work of Hale (2001; 2003; 2006). In this paper, we present novel methods for calculating separate lexical and syntactic surprisal measures from a single incremental parser using a lexicalized PCFG. We also present an approximation to entropy measures that would otherwise be intractable to calculate for a grammar of that size. Empirical results demonstrate the utility of our methods in predicting human reading times.

Original language	English
Pages	324-333
Number of pages	10
DOIs	https://doi.org/10.3115/1699510.1699553
Publication status	Published - 1 Jan 2009
Externally published	Yes
Event	2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009 - Singapore, Singapore Duration: 6 Aug 2009 → 7 Aug 2009

Conference

Conference	2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009
Country/Territory	Singapore
City	Singapore
Period	6/08/09 → 7/08/09

Access to Document

10.3115/1699510.1699553

Cite this

Roark, B., Bachrach, A., Cardenas, C., & Pallier, C. (2009). Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing. 324-333. Paper presented at 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009, Singapore, Singapore. https://doi.org/10.3115/1699510.1699553

@conference{487a88360f4c4e448c22852b312100db,

title = "Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing",

abstract = "A number of recent publications have made use of the incremental output of stochastic parsers to derive measures of high utility for psycholinguistic modeling, following the work of Hale (2001; 2003; 2006). In this paper, we present novel methods for calculating separate lexical and syntactic surprisal measures from a single incremental parser using a lexicalized PCFG. We also present an approximation to entropy measures that would otherwise be intractable to calculate for a grammar of that size. Empirical results demonstrate the utility of our methods in predicting human reading times.",

author = "Brian Roark and Asaf Bachrach and Carlos Cardenas and Christophe Pallier",

year = "2009",

month = jan,

day = "1",

doi = "10.3115/1699510.1699553",

language = "English",

pages = "324--333",

note = "2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009 ; Conference date: 06-08-2009 Through 07-08-2009",

}

Roark, B, Bachrach, A, Cardenas, C & Pallier, C 2009, 'Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing', Paper presented at 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009, Singapore, Singapore, 6/08/09 - 7/08/09 pp. 324-333. https://doi.org/10.3115/1699510.1699553

Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing. / Roark, Brian; Bachrach, Asaf; Cardenas, Carlos et al.
2009. 324-333 Paper presented at 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009, Singapore, Singapore.

Research output: Contribution to conference › Paper › peer-review

TY - CONF

T1 - Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing

AU - Roark, Brian

AU - Bachrach, Asaf

AU - Cardenas, Carlos

AU - Pallier, Christophe

PY - 2009/1/1

Y1 - 2009/1/1

N2 - A number of recent publications have made use of the incremental output of stochastic parsers to derive measures of high utility for psycholinguistic modeling, following the work of Hale (2001; 2003; 2006). In this paper, we present novel methods for calculating separate lexical and syntactic surprisal measures from a single incremental parser using a lexicalized PCFG. We also present an approximation to entropy measures that would otherwise be intractable to calculate for a grammar of that size. Empirical results demonstrate the utility of our methods in predicting human reading times.

AB - A number of recent publications have made use of the incremental output of stochastic parsers to derive measures of high utility for psycholinguistic modeling, following the work of Hale (2001; 2003; 2006). In this paper, we present novel methods for calculating separate lexical and syntactic surprisal measures from a single incremental parser using a lexicalized PCFG. We also present an approximation to entropy measures that would otherwise be intractable to calculate for a grammar of that size. Empirical results demonstrate the utility of our methods in predicting human reading times.

UR - https://www.scopus.com/pages/publications/79952712634

U2 - 10.3115/1699510.1699553

DO - 10.3115/1699510.1699553

M3 - Paper

AN - SCOPUS:79952712634

SP - 324

EP - 333

T2 - 2009 Conference on Empirical Methods in Natural Language Processing, EMNLP 2009, Held in Conjunction with ACL-IJCNLP 2009

Y2 - 6 August 2009 through 7 August 2009

ER -

Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this