(b) From this output, can you answer the question: Is the paceindex statistically significant after adjusting for Region? If not, what do you need to do? (c) From the above output, can you answer the question: Is the Region statistically significant after adjusting for paceindex? If not, what do you need to do? To assess the significance of a set of terms, you want to carry out a "partial $F$-test" by looking at the "drop in SSError" from adding the terms to an existing model. (d) What is the SSError when only paceindex is in the model? ```{r} anova(model1) sum(model1$residuals^2) ``` (e) What is the SSError in the model that includes both paceindex and Region? How much have we reduced the unexplained variation when adding Region to the first model? How many terms did we have to use to get that reduction? ```{r} anova(model3) ``` To decide whether this drop in SSError is statistically significant, we need to compare the value to something. We will use the MSError from the "full model" because we consider it the most accurate estimate of $\sigma^2$. Dividing these terms by their degrees of freedom gives us an $F$-statistic, where the df for the numerator is the difference in the df for the two models. $$ F = {(drop\ in\ SSE)/(difference\ in\ model\ df) \over MSE(full)}$$ (f) Calculate this value from the above output. A short-cut in R that we will make a lot of use of this quarter for comparing "nested" models: ```{r} anova(model1, model3) ``` (g) What do you learn from this output? What do I mean by "nested models"? (h) What do you learn from the following? Is this equivalent to an earlier result? Explain. ```{r} anova(model2, model3) ``` But could we have just run to assess the added contribution of paceindex after adjusting for Region? ```{r} anova(model3) ``` (i) For this output, what hypotheses are tested with the p-value for paceindex?

While the $t$-statistics are always adjusted for all other variables in the model, the $F$-statistics are "sequential" instead. One approach is to change the order in which you enter the variables in the model. ```{r} anova(model4 <- lm(Heart ~ Region + paceindex, data = PaceData)) ``` Another approach is to examine "Type 2" sums of squares. ```{r} #Here I want to use "big A" anova #library(car) car::Anova(model3, type = 2) ``` #### Notes In fact, the residual standard error is used all over the place, e.g., - Smaller \sigma\hat values indicate a better fitting model - 95% prediction interval ≈ $\hat y \pm 2 \hat\sigma$ (outlier tests) - $R^2$ = 1 - SSError/SSETotal $\approx$ $1 - \hat\sigma^2 / s_y^2$ (relative to the variation in the data) - $se(\hat\beta_i) = \hat\sigma /\sqrt{(n-1)Var(X)}$ So it's important that we estimate $\sigma$ "correctly." In particular, the "basic linear model" assumes $Var(\epsilon) = \sigma^2$. If you don't believe you have sufficient heterogeneity, one approach is to make different distributional assumptions. *For next time* Adjusted associations and Interactions