Stat 414 – Review II
Due (preferably before Tuesday, 9am): In Canvas,
(1) post at least one question you have on the material described below.
(2) post at least one example question that you think I could ask on the material described below
Optional Review Session: Interest?
Optional Review Problems: review2probs.html
Format of Exam: The exam will cover Days 1128, Quizzes 1127, Homeworks 36, Ch. 48, and Ch. 10). The exam will mostly be interpretation questions. The exam will be worth approximately 50 points. Be ready to explain your reasoning, including in “laymen’s” language. You can use two pages 8.5x11 pages of notes (hard copy), but I encourage you to prepare as if a closed book exam. Advice: Review Quizzes/commentary, HW solutions/ grader comments, Lecture notes.
Overall
“One can think of mixedeffects
models as ordinary regression models that have additional variance terms for
handling nonindependence due to group membership. The key to mixedeffects
models is to understand how nesting individuals within groups can produce
additional sources of variance (nonindependence) in data.”
From Day 11 (see also Sec 4.7), you should be able to
· Explain the principle of Maximum Likelihood as another possible estimation method
o One variable case
o Two variable case
o How relates to least squares estimation
o How impacts estimation of residual standard error (now considered another parameter simultaneously estimated by the model)
· Using AIC, BIC, and log likelihood values as Information criteria/Measures of Fit
· Using achieved maximum likelihood values to test significance of individual coefficients, groups of coefficients, whole model
o Likelihood ratio test (df)
· Determine
the df used by a model
o Including
if use additional variance terms in the model (e.g., per month)
From Day 12, you should be able to
· Explain
how restricted maximum likelihood (REML) is another possible estimation method
that produces unbiased estimates of .
o Accounts
for the degrees of freedom used to estimate the ’s
o Usually
doesn’t change estimates of ’s
but does change ,
likelihood, AIC, BIC values
o When
sample sizes are large there aren’t major differences among the three
estimation methods
o Be
able to identify which method is being used by the software
· Consider
whether one model is “nested” in another
o Can
you get from one model to the other by setting some of the parameters to 0?
o Can
sometimes still get a “rough comparison” of notnested
models
· Keep
in mind that if use “anova” to compare models, the
models need to be constructed using the same estimation method
o Sometimes
make distinction between whether are comparing models with different fixed
effects or different random effects, but not critical at the moment
· Keep
in mind that weighted least squares is a special case of GLS = generalized
least squares (allows for more interesting variance structures in the data)
· Quiz: Philosophy of maximum likelihood, components of AIC, carry out LRT, chisquare values should be >> df to be significant, df for model/comparing models
From Day 13, you should be able to
·
Identify
whether a study plan is likely to result in correlated observations
o
e.g.,
repeat observations, clustered data, clusterrandomized trials
·
Interpret
ICC calculated from an ANOVA table as a measure of correlation between
observations in the same group
·
Compute
the effective sample size of a study based on the ICC, (common) group size,
number of groups
o
Compromise
between group size and number of groups
·
Discuss
impact on standard errors
From Day 14 (and maybe Sec 3.3), you should be able to
·
Explain the distinction between treating a factor
as random or fixed
·
Discuss when you might choose to treat a
variable as having random effects rather than fixed effects
· Observed “levels” are representative of a larger population (random sample?)
· Want to make inferences to the larger population
· Compute the ICC in terms of random effects (between group vs. total variation)
· Standard errors reflect the randomness at the individual level and from the random effect (e.g., sampling error)
· Discuss advantages such as
o Generalizability
o Degrees of freedom
o Use of Level 2 units + Level 2 variables in same model
From Day 15 (and Sec. 4.8), you should be
able to
· Fit and interpret the “random intercepts” model
o Like adding the “grouping variable” to the model but treating as random effects
o “may be all that is required to adequately account for the nested nature of the grouped data.”
o “there is virtually no downside to estimating mixedeffect models even when If is small or nonsignificant because in these cases the mixedeffect models just return the OLS estimates)” (Bliese et al., 2018)
o Also known as “unconditional means” or “null” model or “oneway random effects ANOVA” (assumes equal group sizes)
· Explain the principle of shrinkage/pooling as it pertains to multilevel models
o where
o No pooling « weights = 1 (treating as fixed effects)
o Complete pooling « weights = 0 (no group to group variation)
·
Predict the impact of pooling on variance
estimates
· Identify and explain factors that impact the size of the weights/consequences/amount of shrinkage
o Determine when (sample size) an estimated mean will be closer to the group mean vs. the overall mean
From Day 16+, you should be able to
·
Write out the random intercepts model: where
·
Discuss consequences of treating a factor as
random effects
o
Impact on model equations (now estimating one
parameter = )
o
Total Var(Y)
= +
o
ICC =
o
VarianceCovariance matrix for the
observations
o
Impact on standard errors of coefficients
(adjusting for dependence)
·
Compute
and interpret intraclass correlation coefficient as expected correlation
between two randomly selected individuals from the same group
o
Show how
this correlation coefficient is “induced” by using the random intercepts model
·
Be able
to explain/interpret the variancecovariance matrix for the responses
Covariance

Site 1 (beach 1) 
Site 2 (beach 1) 
Site 3 (beach 2) 
Site 4 (beach 2) 
Site
1 (beach 1) 


0 
0 
Site 2 (beach 1) 



0 
Site 3 (beach 2) 




Site 4 (beach 2) 
0 
0 


Correlation

Site 1 (beach 1) 
Site 2 (beach 1) 
Site 3 (beach 2) 
Site 4 (beach 2) 
Site
1 (beach 1) 


0 
0 
Site 2 (beach 1) 


0 
0 
Site 3 (beach 2) 
0 
0 


Site 4 (beach 2) 
0 
0 

1 
·
Assess the statistical significance of
the variation among Level 2 units
o
Stating null and alternative hypotheses
o
Fixed effects ANOVA
o
Confidence intervals for variance components
o
Likelihood ratio test (MLE or REML approach)
§
Cut pvalue in half?
·
Understand that we don’t normally make the fixed
vs. random decision based on a likelihood ratio test, but instead consider the
study design/research question of interest
· Identify level 1 and level 2
· Write out the statistical model for a “random intercepts” model (’s, u_{j}, s, )
o Interpret model components
o Define indices
o Interpretation of intercept
o Standard deviations vs. variances vs. covariances
o Composite equation vs. Twolevel equations
· Identify number of units at each level
· Interpret R output (lme, lmer)
o How to use nlme::lme and lme4::lmer in R
o Is output reporting or ?
From Day 17, you should be able to
· Explain the distinction between and
· Add a Level 1 variable to the random intercepts model
o Interpretation of “adjusted” association, parameter estimates
§ Also remember to zero out (intercept) or “fix” Level 2 units (e.g., for a particular school)
o Visualize the model (e.g., parallel lines)
o Level 1 and Level 2 equations, composite equation
o R^{2} type calculations
o Assessing significance (ttest vs. likelihood ratio test)
From Day 18, you
should be able to
·
Explain
the distinction between variation explained by fixed effects vs. by random effects
·
Explain
the distinction between total variation and variation explained by random
effects after adjusting for fixed effects
· Calculate percentage change in Level 1, Level 2, and overall variation after including variable
·
If a
variable coming into the model is related to the level 2 units, variation
explained will depend on the nature of that relationship (similar
to adjusted associations in regular regression models)
· Include a Level 1 variable and reinterpret coefficients as adjusted for that variable
o Also remember to zero out (intercept) or “fix” Level 2 units (e.g., for a particular school)
From Day 19 you should be able to:
· Distinguish Level 1 vs. Level 2 predictors
· Write out “level equations” and “composite” equation using appropriate symbols and indices
o Use estimated equations (fixed and random effects) to predict a response value
· Include a Level 2 variable to (ideally) explain variation in intercepts
o Interpreting an aggregated group mean variable (“if the group mean IQ increases…”)
o Slope coefficients with x and vs. slope coefficients with and
Model includes 
Can explain variation 
x 
Level 1 and Level 2 

Level 2 
x – 
Level 1 (“pure Level 1”) 
x and 
Level 1 (only) and
“emergent” effect at Level 2 
x and 
Level 1 (only) and Level 2
(separated) 
From Day 20 (and Sec 5.1), you should be able to:
· Interpret random slopes models (aka “random coefficients”)
o Interpretation as interaction across higher level units
§ e.g., Level 1 variables can have random slopes at Level 2
· Explain the distinction between a random slopes model and fitting a separate equation for each Level 2 group
o Complete pooling vs. Partial pooling vs. No pooling
· Interpret
the standard deviation/variance of the slopes ()
· Write out the Level 1 and Level 2 equations for the random slopes model
o Including specifying error terms, and their distributions, and covariance terms
o Do be careful with indices
o Thinking of Level 2 equations as “intercepts as outcomes” and “slopes as outcomes” models
o Interpretation of the model components
o Generally include correlation between slopes and intercepts (and slopes and slopes) in the model but can force to be zero
· Compare the random slopes to the fixed slopes models and decide significance of random slopes
· Compare relative sizes of variance components in context
· Interpretation/visualization of variance component for slopes
o 95% of slopes should fall within 2 SD of overall slope
· Interpretation of covariance/correlation between random intercepts and random slopes
· Distinguish between “random slopes” (and “slope effects” (
From Day 21 (and Sec. 5.2), you should be able to:
· Add a Level 2 variable to explain variation in random slopes
o Inclusion of crosslevel interactions
§ Interpretation
o Level equations (adding Level 2 variable to equation for intercept vs. slope vs. both)
o Measuring change in Level 1 and Level 2 variances as percentage change
§ Could explain pretty much all of the Level 2 variation/a Level 2 variable can be sufficient adjusting for the clustering in the study design
No Day 22 Handout
From Day 23 (and Ch. 8) you should be able to:
· Explain how random slopes models induce heteroscedasticity in the responses
o Variance as quadratic function of
o Minimized at
§ Does this occur within the range of values in the dataset?
· Explain how random slopes models different correlations among pairs of depending on the values involved
o Translating between and )
· Interpret covariance/correlation between random intercepts and random slopes
o Distinguish between cov(y’s) and cov(u’s)
o Sign of covariance and implications for fanning in/fanning out of lines
§ Interpretation in context
o Translating between and )
§ Recognizing which is reported in the output
· Interpret the variancecovariance matrix output (marginal vs. conditional)
o Compare model predictions to observed results
· Distinguish between Level 1 variance, Level 2 variance, and total variance
· Explain limitation of ICC in random slopes model
From Day 24, you should be able to:
· Consider random slopes for multiple variables
o Do you want them to be correlated?
o Can increase complexity of model pretty quickly
o Interpretation of random effects correlations (and identifying pairs from output)
· Determine the number of parameters being estimated in a model
o SD and Var are just 1, Include covariances
From Day 25 (and Ch.
10), you should be able to:
· Examine, identify, interpret residuals in a multilevel model
o Marginal vs. Conditional residuals vs. Random effects
o How they compare to each other based on relative position of observations/group
· Explain how patterns in a residual plot could suggest changes to the model (e.g., interaction)
From Day 26, you should be able to:
· Explore and interpret influential observations (what it means to be influential and why we care), possible remedies
o DFBetas vs. Cook’s D vs. Change in significance
o How to interpret graphs from influence package
o Multicollinearity
From Case Study you should be able to:
· Explore
a data set, including graphs for Level 1 and Level 2 relationships and possible
consequences for model building
· Interpretation
of different model components in context
o
Go beyond “variation
in intercepts and slopes” but be able to explain in context what the intercepts
and slopes represent
o Including
for categorical variables
o Including
variance explained
o Including
interactions
o Including
main effects when have interactions
§ When need to “fix” other variables and when need to set them to zero or mean
· Suggest
improvement/next steps to models
o E.g.,
removing insignificant terms and what that means for the R code, the model
equation, and the graphs of the fitted equations
In addition:
o Using estimated effects to make predictions (e.g., “what is the expected response for a student with IQ=100” “What is the expected response for a student with IQ = 100 at school 2”)
o Explain the principle of shrinkage/pooling as it pertains to multilevel models
o
Estimate the precision of the estimated random
effects using estimated variance components and group size
§
o Understand that R stores the estimated effects in ranef(model)and the standard errors (variances) in ranef(model2, condVar=TRUE)
§ Can use this information to make confidence intervals around the random effects, but not usually of interest and certainly shouldn’t be used to make any “causal” claims.
· Using estimated effects to make predictions (e.g., “what is the expected response for school 2” or “what is the expected response for a school at the median”)
o (predicted means vs. estimated effects)
o Use properties of the normal distribution to make predictions of (between or within) group effects (e.g., 16^{th} percentile)
· Comparing models
o Using likelihood ratio test to assess whether a full model is significantly better than a reduced (nested) model
§ Be able to write out hypotheses
o Using AIC, BIC to compare models
o Can also calculate percentage change in variation explained at different levels (pseudoR^{2})
· Do be able to interpret model equations and model output
o Parameters vs. statistics
· Centering
o Why good idea
o Interpretation of coefficients
o Group mean centering vs. Grand mean centering
o Within group vs. Between group regressions
§ Including exploration/explanation for why they could differ
· How fixed effects and random effects can mask each other when build a model sequentially
o Can be difficult to know which modeling approach is better so be sure to explore some different possibilities
More reminders
·
Distinguish between
aggregating and disaggregating data
·
Distinguish between
correlation of observations, correlation of errors, and correlation of slopes
·
Be able to set up a model
based on the research questions
o
Determine number of
parameters
o
Translate R code
into model description
·
Translate between
equations and graphs and output
·
Be able to interpret
interactions in context
Keeping track of variances
· = variation in response variable
· = standard deviation of explanatory (aka predictor) variable
· = variation about regression line/unexplained variation in regression model, variation in response at a particular x value
· = sample to sample variation in regression slope
· = sample to sample variation in estimated group effect (estimated random effect)
·
= sample to sample variation in estimated
predicted value. There are actually two “se fit” values, one for a confidence interval
to predict E(Y) and one for a prediction interval to predict .
The latter can be approximated with ,
but actually depends on etc.
·
variation in intercepts
across Level 2 units
·
: variation in slopes across Level 2 units
·
: covariation between intercepts and slopes (vs. )
·
: correlation between intercepts and slopes
·
: covariation between 1 set
of random slopes and 2^{nd} set of random slopes?
(vs. ) – remember we are now ignoring the variancecovariance matrix
for the fixed regression coefficient (previously the (square roots) of the
diagonal elements had given us . We can get the
variancecovariance matrix for the random effects using getVarCov
on a model fit using nlme::lme with variance components on
diagonal (e.g., slopes and intercepts) and covariances on the offdiagonal.
o
getVarCov(model) vs. getVarCov(model, getVarCov(model1, type =
"conditional")) vs. getVarCov(getVarCov(model1, type = "marginal")