Statistical Inference
As in the previous labs, we ask the same question: How often would we get data like we observed if the null hypothesis was true?
As in Lab 2, we need to consider that when we observe a difference between treatment groups, it is possible that this difference arose from an unlucky random assignment rather than a genuine treatment effect. So now you need to decide whether the observed difference is large enough that it cannot plausibly be explained by the random assignment process alone/luck of the draw/coincidence. Instead of assuming we have a certain number of successes and failures to randomly assign to the treatment groups as in Lab 2, this time we will assume that each student would have received the same memory score, regardless of which treatment group (which grouping of letters) he or she had (this is the null hypothesis). So we will look at the difference in the group means from different simulated (hypothetical) random assignments under the null hypothesis in order to see what pattern the differences in means has when we know there is no actual treatment effect. Then we can see where our results fall in this distribution.
It is very important to keep in mind, that this simulation assumes there is no difference between the two treatments. Then we will examine a randomization distribution of the difference in the treatment means under this null hypothesis that there is no treatment effect, to see whether the difference actually observed in the study is a surprising outcome under the null hypothesis. (If so, we reject the null hypothesis.)
