**Stat 301 – Review 2**

* *

**Due Monday, midnight:** Submit Exam 2 Question and Exam 2 Example
Questions in Canvas and then review responses before exam

**Optional Review Session:** Tuesday, 7-8pm, Zoom office hours link in
Canvas

**Exam Format:*** *The exam will cover topics from Chapter 2 – *one
quantitative variable *and Chapter 3 – *comparing two groups on
a binary response variable* (Weeks 5-7, Monday, HW 4-6). The exam
format will be similar to Exam 1 with a mix of multiple choice, short answer,
and longer answer questions. You will not be using software or applets but you
will be expected to interpret provided output (e.g., from R, JMP, applets,
other), discuss how you would use technology (e.g., R or JMP or applet), and
interpret “code.” You may use two pages of your own notes (8.5 x 11, front and
back). You should also **have your calculator** (not a cell
phone). The questions will not be heavily computational, but you are expected
to know how to set up the calculations by hand. You are also expected to
explain your reasoning, show your steps, and interpret your results. The exam
will contain approximately 50 points (so a 2-point problem should only take you
about 2 minutes).

**Study Hints: **You should study from the text (including
chapter summaries), class meetings, investigation solutions (see links on main
Canvas page), powerpoint slides, labs, homeworks, homework solutions,

** **

**Review Problems:** Click here (Solutions)

**Principles to keep in mind:**

Standardized/test statistic = (statistic –
hypothesized)/(SE of statistic)

Confidence interval = observed statistic __+__ (critical
value) × (SE of statistic)

SE of statistic estimates sample to sample
variation of statistic (vs. variability in data)

**From Chapter 2**

The big ideas in Chapter 2 are essentially the
same as Chapter 1, just focused on the mean as the parameter of interest rather
than a proportion. “Descriptive statistics” are more interesting, and you should
be able to discuss which graphs/values are more informative in different
situations and which comparisons are most useful (e.g., a comparison of centers
wasn’t helpful in answering the cancer pamphlet question). The only change we
make when we use the sample mean as the statistic is to use the *t*-distribution
as our ‘reference distribution’ rather than the normal distribution. One new
idea is a “prediction interval” = a confidence interval for an individual
(e.g., future) observation. With
proportions, we assumed every individual observation had the same probability,
that is, no person to person variation.
The person-to-person variation with a quantitative variable is
summarized by (in the
population) and *s* (in the sample).
The validity conditions for the central limit theorem for the normality
of the sampling distribution of the sample means also changed formed slightly.

**From Section 2.1 (Investigations 2.1, 2.2, 2.3)
you should be able to:**

·
Anticipate the behavior of a variable based on
context

·
Create and interpret different types of graphs: histogram, dotplot, stemplot, boxplot

o Critique effectiveness of display (e.g., labels,
bin width, hiding patterns and important features)

·
Describe the behavior of a variable’s distribution from the graph:
shape, center, variability, unusual observations

·
Tell me where data are rather than where they are not

·
Describe the shape of a distribution of a quantitative variable as
symmetric, skewed to the left, or skewed to the right

·
Assess the normality of a distribution (e.g., overlay curve on
histogram, is normal probability plot linear)

· Identify possible outlier(s) in the dataset
(visually, 1.5IQR criteria) and suggest explanations based on context

· Critique justifications for removing outlier(s)
from dataset

· Interpret the five-number summary and the
inter-quartile range (IQR)

· Understand how skewness and/or outliers impact
the relative positions of the mean and median and the values of the standard
deviation and IQR

· Explore data transformations to normalize a
distribution

· Transform data and use a normal model to
estimate a probability

**From Section 2.2 (Investigations 2.4, 2.5, 2.6)
you should be able to:**

· Continue to consider whether or not you are
likely to have a representative sample

· Explain the reasoning behind simulating random
samples from a finite population

o Critique assumptions made about the population

· Predict the behavior of the sampling
distribution of the sample mean (mean, standard deviation, shape) and compare
to the population distribution

o How/when does the shape of the population
matter?

o Distinguish between (i) the population, (ii) a
sample, and (iii) the sampling distribution

§ Larger sample sizes impact the shape and variation of the sampling
distribution but NOT the population or sample

· Apply the Central Limit Theorem of the sample
mean

o Know when it does/does not apply

o “Use technology” (R, JMP, or Normal Probability
Calculator) to approximate probabilities for sample means with the
normal distribution

§ What values to use for mean, SD, observation; direction

§ Sketch, label the corresponding distribution and shade the
probability of interest

§ Interpret provided output

· Define a population mean in
context

· State appropriate null and alternative
hypotheses about a population mean for a given research question

· Calculate and interpret the standard error of
the sample mean, *s*/.

· Determine and interpret the “standardized”
distance between and

· Roughly approximate a 95% confidence interval
for by

· Use the *t*-distribution to model the
behavior of the standardized statistic

o Determine the degrees of freedom (sample size
minus one) and the impact on the *t* distribution

§ As sample size increases, df
increases, *t* distribution approaches standard normal
distribution

o Explain the difference between the normal
distribution and the *t* distribution

§ Heavier tails

§ Why that’s helpful to use the *t*-distribution for
inference about the population mean

§ Consequences on p-values, confidence intervals, coverage rate

o Assess the validity of the *t* procedures

· Interpret a confidence interval for

o If asked, interpret the confidence level

· Determine and interpret a prediction interval
(PI) for a future observation

o With raw data, by hand (e.g., or roughly roughly )

o Explain the reasoning behind the SE formula for
a PI

o Compare a confidence interval to a prediction
interval

o Assess the validity of the prediction interval
procedure

~~From Section 2.3 you should be able to:~~

~~·~~~~ ~~~~Take a log transformation, apply a ~~*t* confidence
interval, back-transform the endpoints of the interval to the original
measurement units

** **

**From Chapter 3**

In Ch. 3, the big picture to keep in mind is
comparing a categorical response variable between two groups and that it
matters a lot how those groups were formed: an observational study with
independent random samples or a randomized experiment. This
distinction must be considered when drawing your final conclusions (can I
generalize to the larger populations, can I draw cause and effect?), and should
probably also be considered when you analyze the data (are modeling random
samples from populations or random assignment). This can impact the standard
errors that you use, but with large sample sizes the results won’t differ too
much, and analysts tend to apply the same normal approximation.

** **

**From Section 3.1 (Investigations 3.1 and 3.2)
you should be able to:**

· Construct a two-way table of counts (explanatory
variable as columns)

· Calculate (appropriate) conditional proportions
and compare them

o Proportion of 6 ft tall men in the NBA vs.
Proportion of NBA players over 6 ft vs. proportion of men that are 6 foot tall
NBA players. (*Hint*: Follow the ‘of’)

· Create a segmented bar graph from a two-way
table ~~(may use technology) ~~and describe what it reveals (e.g.,
do the distributions differ across the groups)

· Define the parameter in terms of the difference
in population proportions

· State hypotheses in terms of the difference in
population proportions

· Simulate random sampling (independent binomials)
from two (large) populations under the null hypothesis

o Create a null distribution of differences in
sample proportions

o Interpret graphical and numerical summaries of
this distribution

o Estimate or obtain a p-value from simulation
results

o Explain the simulation process (e.g.,
independent random samples with same probability of success)

o Interpret the p-value in context (e.g., X% of
random samples…)

· Determine whether a normal approximation to the
null distribution should be valid

o Remember the simple way of checking this is all cell
counts in table are at least 5, list the values you are looking at

o Should also consider sizes of (finite)
populations sampling from and whether they are more than 20 times the sizes of
the samples

o Reasoning behind the standard error formula (adding
variances)

· Pooled vs. unpooled
variance estimates (test statistic vs. CI)

· Calculate a *z *test statistic
and p-value using the normal distribution

o Interpret the standard error, test statistic,
and p-value in context

· Calculate and interpret a *z-*confidence
interval for the difference in two population proportions

o Make sure the direction is clear. Go beyond
saying is in the
interval, but in terms of how much higher/lower is than (in
context)

· Discuss factors that will affect standard error,
test statistic, p-value, confidence interval, and how

o e.g., sample size, order of subtraction, size of
difference in sample proportions

· Distinguish between the explanatory variable and
the response variable from a study description

· Identify and explain a potential confounding
variable in observational studies

o Be sure to explain on how there could be a
differential effect by the confounding variable on the response variable
between the explanatory variable groups (Make sure it’s an alternative
explanation for the observed difference between groups separate from the
explanatory variable, not just another variable or a feature that applies
equally to both groups).

** **

**From Section 3.2 (Investigations 3.3, 3.4) you
should be able to:**

· Distinguish between an observational study and
an experimental study

o Be able to justify which type of study you have

o Be able to critique advantages and disadvantages
of different study designs

· Discuss the advantages of using a placebo
treatment

· Discuss the advantages to *blinding* and *double-blinding* in
a study

· Discuss the purposes/goals/merits of
“randomization” (aka random assignment to treatment groups)

· Identify when we are allowed to draw
cause-and-effect conclusions (perhaps just about the experimental units in the study)

· Interpret and critique a description of a
research study (e.g., Inv 3.4)

· Discuss some of the limitations in the type of
conclusions that can be drawn from different designs

o Do not draw cause-and-effect conclusions from an
observational study

· Can still decide whether there is evidence of an
association, measure how strong the association is

· Identify and justify the appropriate “scope of
conclusions” (generalizability, causation) from the study design

o Table on p. 188 is the best in the book!

**From Section 3.3 (Investigations 3.5, 3.6, 3.7)
you should be able to:**

· Define the parameter in terms of the difference
in (long-run) treatment probabilities

· Simulate random assignment under the null
hypothesis, create a null (or randomization) distribution of the difference in
two sample proportions

o Explain the reasoning behind randomization test
(fixing number of successes and failures models “no effect” from treatment
group assignment)

o Carry out and interpret the results from a
randomization simulation for a two-way table (e.g., dealing poker chips,
including how many poker chips and how many of each color, how many deal out to
each group)

o Use the Analyzing Two-way Tables applet

o Understand the equivalence of using the number
of successes in group A, difference in group proportions, relative risk, and
odds ratio as the statistic in this simulation

o Including how to approximate the (one or two-sided)
p-value based on the simulation results

§ Can double one-sided p-value if distribution is symmetric, use
method of small p-values otherwise

o Interpret the p-value in context (e.g., X% of
random shuffles…)

· Calculate the exact (one or two-sided) p-value
using the hypergeometric distribution (aka Fisher’s Exact Test)

o Including showing set up by hand and with
technology

o Including writing out the probability statement
P(X __>__ …. ) and the input values of the hypergeometric (*N,
M*, *n*)

o “Using technology” to carry out the full FET
p-value calculation

· Approximate (and interpret) the p-value and
confidence interval for using the normal distribution
(two-sample *z*-procedures)

o Decide whether the *z*-procedure is valid
(just worry about cell counts, not population size)

·
Consider continuity correction for p-value (half-way to next
possible statistic outcome)

·
Consider Wilson adjustment/Plus Four adjustment (adding 1 to each
cell in the table) as an improvement for the confidence interval

**From Section 3.4 (Investigation 3.8, 3.9, 3.10)
you should be able to:**

· Calculate and interpret relative risk as an
alternative measure of association between two binary variables

o Remember that the difference in proportions does
not take into account the magnitude of the baseline risk

§ Small differences in proportions “seem” much larger when the
baseline risk is small

· Simulate a null distribution (using random
sampling and/or random assignment) under the null hypothesis and interpret the
results using the relative risk as the statistic

o Create a null distribution of relative risk

o Including how to approximate the p-value based
on the simulation results

· Determine (by hand and with applet) and
interpret a confidence interval for the ratio of treatment probabilities using the normal distribution

o Including how and why we “transformed” the
statistic to log relative risk

o Calculate and interpret the standard error of
the transformed statistic

·
Back transform and interpret the confidence interval in context

o
Make sure direction is clear

·
Distinguish between a cohort, case-control, and cross-classified designs
of an observational study and how the design affects which numerical summaries
you can reasonably interpret

o Cohort: sample based on EV; Case-control: sample
based on RV; Cross-classified: Sample and ask two questions

o Don’t use relative risk or difference in
proportions with case-control studies

o It is always ok to calculate odds ratio

· Calculate and interpret odds ratio as an
alternative measure of the association between two binary variables

o How to decide which calculation is being asked
for in the context of the problem (how define success, group A)

o How to interpret the results of the calculations

· Interpret
a confidence interval for the population odds ratio in context

·
Make sure direction is clear

** **

**Coding principles you should be able to
interpret/explain/write pseudo-code**

· Subsetting data

· Recoding a categorical variable

· Splitting the graph by an explanatory variable

· Create simulations to replicate random sampling
and/or random assignment

· You should also be able to interpret generic
computer output for the procedures we have learned (one-sample *t*-procedures,
two-sample *z*-procedures)

** **

**What you should be able to do with the
calculator**

·
Calculate conditional proportions, relative risk, odds ratio

·
Approximate 95% confidence intervals for population mean, next
observation

·
Calculate confidence intervals for relative risk, odds ratio

** **

**Things you need to remember from Exam 1**

·
Defining observational units, variables, and parameters in context

·
How to interpret probability as a long-run relative frequency

·
Showing your work/explaining how would use the computer

·
Explaining your simulation process

·
One-sided vs. two-sided alternatives

·
The reasoning of statistical significance and what a p-value
measures

·
Making conclusions based on the size of the p-value (remember to
provide “linkage” between your p-value and conclusion)

·
Interpreting confidence intervals and confidence levels

·
“Duality” between confidence intervals and tests of significance

·
The concept of power and factors that affect power

·
Comparing “theoretical” and “simulation” results

·
Margin-of-error measures sample to sample variation (due to random
sampling) but does not account for any “nonsampling
errors” (e.g., poorly worded questions)

**Keep in mind**

·
When to talk in terms of population means, μ, and when to
talk in terms of probabilities,

·
When comparing distributions, remember to cite your evidence if
you think there is a difference in the groups. In particular, tell me what you
see in the summary statistics (e.g., a higher proportion) that leads to your
conclusion (e.g., abstainers more likely to develop peanut allergy than
consumers)

·
Remember that the confidence *level* refers to the
reliability of the method – how often, in the long run, random samples (or
random shuffles) will produce an interval that succeeds in capturing the
population parameter

·
Remember to think about the direction of subtraction used by the
technology

·
We can use a one-sample *t-*procedure even when the
sample sizes are small if we have reason to believe the population distribution
is normally distributed. You can try to judge this, especially if you don’t
have past experience with the variable, based on graphs of the sample
data. If the sample data looks reasonably normally distributed
(normal probability plots are a useful tool for helping this judgment), you can
cite this as evidence that the population distribution is normally distributed.
If you aren’t sure, then use an alternative analysis instead (e.g., data
transformation, simulation like bootstrapping).

·
Keep in mind the one-sample *t*-procedure only tells you
about the population mean (vs. other aspects of the distributions)

·
Always putting your conclusions in the context of the research
study

·
Including considering “practical significance” (could the
difference have happened by random chance alone, is the difference considered
meaningful in the context of the variable, e.g., is a .5 ^{0}F
difference likely to matter)

·
Try to avoid the word “accurate” without explaining exactly what
you mean by it.

·
Always try to say the distribution *of what*

·
Try to avoid use of the word “group” but clarify if you mean the
sample or the population or the long-run treatment

·
Avoid use of the word “it”

** **

**Also keep in mind:**

·
Part of your grade will be based on communication. Be
precise in your statements and use of terminology. Avoid unclear
statements, and especially don’t use the word “it”! Always relate your comments
to the study context.

·
Show the details of any of your calculations.

·
Organize your notes ahead of time, and don’t plan to rely on your
notes too much.

·
Be able to *both* make conclusions from a
p-value *and* provide a detailed interpretation of what the
p-value measures in context

o
Improving interpretations of p-values:

§
Random chance = random sampling or random assignment

§
Alone = null hypothesis

§
Observed result = cite value of statistic from study

§
Or more extreme = give direction(s)

·
Keep in mind that “statistical significance” is an adjective of
the sample data or the statistic, NOT the population parameter

·
You should continue to focus on the overall statistical process
from collecting the data, to looking at the data, to analyzing the data, to
interpreting results

·
When stating final conclusions, cite the specific evidence (e.g.,
it is/is not statistically significant *because* my p-value of
XXX is small/not small)

·
Simulation-based vs. Exact vs. Theory-based (normal) procedures

·
Think big picture and be able to apply your knowledge to new
situations

** **

**Some additional Lessons from HW**

__HW 4__

·
Subsetting data and possible consequences on scope of
conclusions

·
Identifying outliers by 1.5IQR criteria

·
Behavior of mean vs. median/what each measures about a
distribution (and what they don’t measure)

o Benefits of mean over median
for inference

o Limitations of mean vs.
median

·
Always interpret your results in context

·
Sample standard deviation as a measure of “typical prediction
error” when using the mean

·
Applying a data transformation

·
Sample vs. Population vs. Sampling Distribution and what impacts
the behavior of each

__HW 5__

·
Be specific to the study context vs. generic statements (e.g., to
guard against carry-over effects vs. to avoid bias)

·
(Lack of) role of population shape when sample size is large

·
Be able to state hypotheses both in using symbols and
words. Make sure you define the symbols. Make sure you
clarify what number is being tested.

o
I find the phrase “true value” unclear, and instead would talk in
terms of “population mean” – keeping in mind that all a test of significance
can make conclusions about is the mean, not individual observations. For
example, all we could conclude in the Facebook study was that the times tended
to be longer than Instagram or were longer on average (rather than Facebook
‘always’ was longer than Instagram)

·
If asked to “estimate a parameter” – use a confidence interval,
not just the sample statistic

o
If you are interpreting an interval for a difference *be very
clear* what you think is larger than what

·
Remember the validity conditions for using *t*-procedures are
“either or”

o
Though a prediction interval requires normality, large sample size
doesn’t solve that requirement

·
Be able to distinguish (in interpretation, in identification)
differences between a confidence interval for a mean and a prediction interval

·
Bootstrapping is an alternative approach for estimating sample to
sample variation *in any statistic*

o
Goal of our simulation/theory-based methods is to estimate the
“chance variation” to help us determine how far our statistic could plausibly
be from the parameter of interest

·
‘Effect sizes’ are often used as a measure of “practical” rather
than statistical significance

__HW 6__

·
Justify “experiment” by considering “active imposition of
explanatory variable”

·
Using the research context to formulate a one or two-sided alternative
hypothesis

·
Matching the simulation model to how randomness was used in the
study design (and why it might matter)

o
Interpreting the p-value accordingly (by this point in the course,
your interpretation should now be beyond “by chance alone” – you need to
explain the source of randomness and the assumptions of the null hypothesis)

·
Justifying conclusions

o
Significance: p-value

o
Estimation: confidence interval

o
Causation: randomized experiment (and significant)?

o
Generalizability: random sample?

**Applets**

I will assume you are familiar with the
output/functionality of these applets for Ch.
2 & 3.

·
Descriptive Statistics (mean, median, SD, IQR for quantitative
data, possibly across groups, boxplots, histograms, dotplots,
normal probability plot, time plot)

·
Sampling from a Finite Population (can input a large population of
values to random sample from, view individual samples, generate a sampling
distribution for mean, median, t-statistic)

·
Normal Probability Calculator (can specify variable, mean, SD, and
region of interest to find probability or probability to find region or z-score
to find probability and region)

·
Simulating Confidence Intervals (can explore behavior of different
interval procedures to see long-run coverage rate)

·
t Probability Calculator (can specify df
and region of interest to find probability or probability to find t* (critical
value))

·
Theory-Based Inference (can conduct one and two-sample tests and
confidence intervals for proportion and means)

·
Comparing Two Population Proportions (can simulate independent
random samples from binomial processes, examine individual sampling distributions
and sampling distribution of difference in proportions)

·
Randomizing Subjects (can explore “balance” created between groups
with random assignment) – see Investigation 3.3

·
Analyzing Two-way Tables (given 2x2 two-way table, can find
simulation-based p-value, Fisher’s Exact Test, normal approximation, 95%
confidence intervals for difference in probabilities, relative risk, and odds
ratio).