U of T linguistics professors Shohini Bhattasali and Dave Kush have had articles published in the Journal of Memory and Language, as part of their special issue on Language Models and Psycholinguistics: What can one black box tell us about the other?
The abstracts of both papers can be found below:
Kobzeva, A., Arehalli, S., Linzen, T., & Kush, D. (2025). Learning filler-gap dependencies with neural language models: Testing island sensitivity in Norwegian and English. Journal of Memory and Language, 144, 104663.
Human linguistic input is often claimed to be impoverished with respect to linguistic evidence for complex structural generalizations that children induce. The field of language acquisition is currently debating the ability of various learning algorithms to accurately derive target generalizations from the input. A growing body of research explores whether Neural Language Models (NLMs) can induce human-like generalizations about filler-gap dependencies (FGDs) in English, including island constraints on their distribution. Based on positive results for select test cases, some authors have argued that the relevant generalizations can be learned without domain-specific learning biases (Wilcox et al., 2023), though other researchers dispute this conclusion ((Lan et al., 2024b; Howitt et al.,2024). Previous work focuses solely on English, but broader claims about filler-gap dependency learnability can only be made based on multiple languages and dependency types. To address this gap, we compare the ability of NLMs to learn restrictions on FGDs in English and Norwegian. Our results are mixed: they show that although these models acquire some sophisticated generalizations about filler-gap dependencies in the two languages, their generalizations still diverge from those of humans. When tested on structurally complex environments, the models sometimes adopt narrower generalizations than humans do or overgeneralize beyond their input in non-human-like ways. We conclude that current evidence does not support the claim that FGDs and island constraints on them can be learned without domain-specific biases.
Ohams, C., Nair, S., Bhattasali, S., & Resnik, P. (2026). A predictive coding model for online sentence processing. Journal of Memory and Language, 146, 104705.
Computational approaches to prediction in online sentence processing tend to be dominated by computation-level surprisal theory, offering few insights into underlying cognitive mechanisms. Conversely, predictive coding is an algorithmic theory grounded in neuroscience, but it has rarely been employed in the study of language processing, in part because its areas of application have not involved sequential processing. Building on a recently proposed temporal predictive coding model, we present what is to our knowledge the first exploration of sequential predictive coding in broad-coverage online sentence processing. We investigate our model at non-toy scale using naturally occurring language, establishing its cognitive validity via comparison with reading times, and we link measurable aspects of the model to cognitive discussions of mechanism for prediction in language processing. Our results suggest that sequential predictive coding models are a valuable complement to surprisal theory as a route to progress on process-oriented theories of language comprehension.
Congratulations to professors Bhattasali and Kush!