- Dates: 12-16 September 2022.
- Times: 9AM-5PM daily.
- Location: The summer school will be held at the Griebnitzsee campus in Potsdam, at Haus 6. For train connections, consult bvg.de; the train station near the campus is called Griebnitzsee Bhf.
**Application period: 17 Sept 2021 to 1 April 2022**. Applications are closed. Decisions will be announced around 15 April 2022.

- Prof. Dr. Lena Jäger, Zürich, Switzerland. (Tuesday, 13 Sept 2022, 5-6PM).

Title: Bayesian Estimation of Measurement Reliability of Individual Differences in Sentence Processing

Abstract:

Theories of human sentence processing generally assume that the cognitive mechanisms involved in language processing are qualitatively identical across speakers. However, over the past decade, evidence has accumulated indicating that individual differences in a comprehender’s cognitive capacities play an important role in sentence processing (e.g., Vuong and Martin, 2014; Nicenboim et al., 2015; Farmer et al., 2017). From a methodological point of view, the first step for a principled investigation of individual differences in sentence processing is to establish their test-retest measurement reliability, that is, the correlation of subject-level effects across multiple experimental sessions (Parsons et al., 2019; Cunnings and Fujita, 2020). We can’t take this measurement reliability as a given because of the so-called reliability paradox which states that test-retest measurement reliability at the individual level is necessarily lower for manipulations with high between-subjects reliability, that is, replicability at the group level (Hedge, 2017). However, it is likely that precisely effects with high replicability at the group level constitute the set of well-established psycholinguistic phenomena which build the foundation of sentence processing theories. In this talk, I will present ongoing work in which we assess the measurement reliability of individual differences in a range of theoretically relevant phenomena within and across methods. We collected the first cross-methodological reading corpus with multiple experimental sessions from each participant. 50 native speakers of German each participated in four experimental sessions (two eye-tracking and two self-paced reading sessions, 200 sessions in total). Participants read 80 pages of natural text (20 pages per session), and participated in a comprehensive psychometric assessment measuring verbal and non-verbal working memory capacity, cognitive control and IQ, as well as lexical and non-lexical reading fluency. We estimate within- and cross-methodological test-retest measurement reliability of individual differences in well-established psycholinguistic effects by computing Bayesian correlation estimates (Matzke et al., 2017) of participant-specific random slopes between sessions from the same method, and sessions from different methods, respectively. We further explore whether cognitive capacities affect test-retest measurement reliability of individual-level effects. We find that lexical-level effects are very stable within individuals across sessions and methods (e.g., participants exhibiting a particularly strong word length effect do so across sessions and methods). By contrast, higher-level cognitive effects that involve syntactic processing (e.g., surprisal or dependency locality) are less stable within individuals. In a nutshell, we find that for higher-level effects, test-retest measurement reliability of individual-level effects is generally higher in self-paced reading than in eye-tracking. Cross-methodological measurement reliability of individual-level effects is generally low for all eye-tracking measures. Future work will need to address the question whether the observed low test-retest measurement reliability of higher-level cognitive effects can be explained by the stimulus materials (naturalistic texts) which, in contrast to minimal pairs of planned experiments, do not push the comprehender’s sentence processor to its limits, and therefore might be less adequate to assess individual differences in sentence processing. - Prof. Dr. Riccardo Fusaroli. (Thursday, 15 Sept 2022, 5-6PM).

Title: Standing on the shoulders of normal-sized people. Promise and challenges of cumulative statistical approaches

Abstract:

We often hear that Newton stood on the shoulders of giants and that science is a cumulative enterprise where new research builds on previous results. This conception of science relates to a commonly cited benefit of Bayesian approaches: their ability to integrate diverse sources of information, e.g. results of previous studies as informed priors. However, this practice is rarely seen in the literature. One possible explanation could be that we remain skeptic of scientific findings in our own field; that is, we know that we stand on the shoulders of normal-sized, fallible people (just as we ourselves are), rather than on the shoulders of giants. This raises the question of how we best integrate fallible findings from previous analyses in our studies. In this talk I will tackle this issue using a combination of simplified simulations, and the concerns arisen in concrete studies using informed priors. First I will cover simulation-based studies of posterior passing: what happens when we use previous posterior estimates as priors, in a sterilized in silico environment? I will then let the complexity of real research slowly creep in: from linear chains of one study following the other, to interrupted chains due to publication bias, to meandering forking paths where studies know and include only some of the literature. These simulations show that posterior passing is slowed down by complexity, but still provides the best solution for this cumulative enterprise. With the simulations at hand, I will turn to real application scenarios, where previous literature and expert opinions are used to build informed priors. Novel concerns arise: hierarchical structures of expectations, heterogeneity of studies, undue levels of confidence, etc. Based on these results, I will advocate for a critical use of informed priors, involving comparisons between informed priors and alternative (e.g. skeptical) priors, and explicit testing of inferential robustness.

We offer foundational/introductory and advanced courses in Bayesian and frequentist statistics. When applying, participants are expected to choose only one stream. This year, there will be a special series of lectures by Ralf Engbert that everyone is welcome to attend.

**Special short course: Introduction to Dynamical Models in Cognitive Science**(all participants are welcome, no need to register). Taught by Ralf Engbert, assisted by Lisa Schwetlick and Maximilian Rabe. Tuesday and Thursday, 3:00-4:30PM. Location: Hoersaal 03.

This course is an introduction to dynamical modeling of eye movements during reading. Lecture I (Tuesday) starts with an introduction to the basic concepts of mathematical modeling using ordinary differential equations. We develop a simplified version of the SWIFT model for eye guidance in reading (Engbert et al., 2005; Rabe et al., 2021), including the computer implementation with R. Lecture II (Thursday) introduces sequential likelihood methods for dynamical processes. We show that the likelihood function can be decomposed into temporal and spatial components. For the simplified SWIFT model, we carry out numerical computations of the likelihood function.

Course material: All slides and computer code will be made available via OSF at https://osf.io/8wrf6/

Timing: Tuesday and Thursday: 3:00-4:30PM.

**Introduction to Bayesian data analysis**(maximum 30 participants). Taught by Shravan Vasishth, assisted by Anna Laurinavichyute.
This course is an introduction to Bayesian modeling, oriented towards linguists and psychologists. Topics to be covered: Introduction to Bayesian data analysis, Linear Modeling, Hierarchical Models. We will cover these topics within the context of an applied Bayesian workflow that includes exploratory data analysis, model fitting, and model checking using simulation. Participants are expected to be familiar with R, and must have some experience in data analysis, particularly with the R library lme4.**Advanced Bayesian data analysis**(maximum 30 participants). Taught by Bruno Nicenboim, assisted by Himanshu Yadav
This course assumes that participants have some experience in Bayesian modeling already using brms and want to transition to Stan to learn more advanced methods and start building simple computational cognitive models. Participants should have worked through or be familiar with the material in the first five chapters of our book draft: Introduction to Bayesian Data Analysis for Cognitive Science. In this course, we will cover Parts III to V of our book draft: model comparison using Bayes factors and k-fold cross validation, introduction and relatively advanced models with Stan, and simple computational cognitive models.
**Foundational methods in frequentist statistics**(maximum 30 participants). Taught by Audrey Buerki, Daniel Schad, and João Veríssimo.
Participants will be expected to have used linear mixed models before, to the level of the textbook by Winter (2019, Statistics for Linguists), and want to acquire a deeper knowledge of frequentist foundations, and understand the linear mixed modeling framework more deeply. Participants are also expected to have fit multiple regressions. We will cover model selection, contrast coding, with a heavy emphasis on simulations to compute power and to understand what the model implies. We will work on (at least some of) the participants' own datasets. **Advanced methods in frequentist statistics with Julia**(maximum 30 participants). Taught by Reinhold Kliegl, Phillip Alday, and (over zoom:) Doug Bates.

Textbook: here. We will work through the first six chapters.