3.9 Exercises

Exercise 3.1 By-subjects t-test in a two-condition design

Load the data-set below:

data("df_gibsonwu")

The data consist of a repeated measures experiment (self-paced reading) involving Chinese relative clauses. The experiment compares two conditions which are labeled obj-ext and subj-ext. The column subj refers to subject id, and the column item refers to item id. The column labeled rt refers to reading time in milliseconds at a critical word in the sentence. This is a standard Latin square design.

Using rt as a dependent variable, carry out the appropriate by-subjects and by-items t-test to evaluate the null hypothesis that there is no difference between the two conditions. Write down all the R commands needed to do the appropriate t-tests, and the resulting t-values and p-values. State whether we can reject the null hypothesis given the results of the t-tests; explain why.

Exercise 3.2 A linear mixed model for the two-condition design

Now, fit a linear mixed model (call it M1). Recode the column called Type as sum contrasts (\(\pm 0.5\)).

Assume varying intercepts for subjects and varying intercepts for items (varying intercepts are sometimes called random intercepts). Write down the linear mixed models command, and write down the fixed-effects estimates (intercept and slope) along with their standard errors. State whether we can reject the null hypothesis given the results of the t-value shown in the linear mixed model output; explain why.

Exercise 3.3 Raw RTs vs. Log RTs

Repeat the t-test and the linear mixed model with log reading times as the dependent measure, instead of raw reading times. Does the conclusion change compared to the analyses using raw reading times?

Exercise 3.4 Power analysis on the Gibson and Wu data

The researcher wants to achieve 80% statistical power in a future study that replicates the Gibson and Wu study. Based on the available data above, she determines that the standard error (note: not the standard deviation!) of the difference in means between the two conditions is 47. She has reason to believe that the true difference in means is \(30\) ms. What is the number of participants needed to achieve approximately 80% power? Use the power.t.test function to compute your answer. Write down the power.t.test function specification you used, as well as the number of participants needed, based on the output of the power.t.test function.

Exercise 3.5 Residuals in a linear mixed model for the two-condition design

The plot below shows the distribution of the residuals from model M (fit on raw reading times) plotted against the standard normal distribution with mean 0 and standard deviation 1. Explain what the plot tells us about one of the model assumptions of the linear mixed model M that we fit earlier.

(You can ignore the numbers below the plot.)

Exercise 3.6 Understanding contrast coding

Using only your estimates of the intercept and the slope in model M’s fixed effects output (the model fit on raw reading times), write down the mean of the condition labeled obj-ext in the data, and the mean of the condition labeled subj-ext.

Exercise 3.7 Understanding the fixed-effects output

Suppose that the model M’s output for the fixed effects analysis were as follows. so is the sum-coded contrast specification shown above.

results

##             Estimate Std. Error t value
## (Intercept)    548.4      51.56   10.64
## so             120.4         NA    2.00

What is the value of the standard error of the slope (so), which is labeled NA above?

Exercise 3.8 Understanding the null hypothesis test

A researcher fits a linear mixed model a new data set to compare the reading times between two conditions (a) and (b), just like in the above study. Her hypothesis is that the mean for condition (a) is larger than the mean for (b). She observes that condition (a) has sample mean 500 ms, and condition (b) has sample mean 450 ms. She also establishes from the linear mixed model that the t-value is 1.94. The approximate p-value associated with this t-value is 0.052. Answer the following: (A) Do we have evidence against the null hypothesis, and (B) do we have evidence for the particular research hypothesis that the researcher has?

The researcher runs the same analysis as above on a new data-set that has the same design as above, and now gets a p-value of 0.001. What is the approximate t-value here (assume that the standard normal distribution can be used to approximate the t-value)? Now she has stronger evidence than in the above case where the p-value was 0.052. What does she have stronger evidence for? Does she have stronger evidence for the null hypothesis, or stronger evidence in favor of her particular research hypothesis (or both)?