Use of Exams Are "exams" a component of your students' course grade? What percentage of the overall grade are from exams? What is the format of your main exams? Would you characterize these exams as mostly multiple choice, short answer, longer answer questions, or some mixture?
Are these in-class exams or take-home exams? What time restrictions do you place on the exams? How many students do you expect to finish in the time allotted?

Why do you use exams? Do you feel exams are an important reflection of students' abilities?

Do you take any safeguards to try to minimize student cheating? Please describe the concern and how you respond (e.g., seating arrangements, multiple versions of exams).

Background. I've probably taught this course 20 times. The first several times, I gave an hour-long mid-term and a two-hour final. After the first few times, I started assigning a term project also. For the last several years, I've dropped the final, and instituted about 10 once-a-week quizzes. The hour-long mid-term and project remain, and students also give 5-minute oral presentations based on their projects at the end of the semester. I think the weights I used last time was weekly HW 30%, quizzes 20%, mid-term exam 20%, project 30%.

Timing issues. Although the mid-term exam is nominally a one-hour exam, I try to find additional classrooms, and allow students to come early and/or stay late, in order to provide extra time for those who want it. To the extent practical, and consistent with ensuring that no students feel unfairly deprived, I want each person to have as much time as she needs, although sometimes scheduling makes this hard to accomplish. Almost all students finish the exam. Typically a few say they didn't have enough time, but their papers generally show that they didn't have a strong understanding of the material, and worked slowly because they had to guess at what to do as they went along, and were relying on the book to learn things that they should have known already. However, I sometimes decide that a particular student would have done better with additional time, and in that case I make a one-of-a-kind adjustment in my grading. (For a clear-cut example, think of a student who gives near-perfect solutions to the first 8 problems, working them in order, and leaves the last 2 blank.)

Cheating. Fortunately, this is not an issue for me. We have an honor code, which seems to work well. I talk briefly (but earnestly!) about the honor code in class in advance of the exam, but don't do anything else. Typically, I'm not in the room during the exam, except a few times when I come back to see if anyone has questions.

Rationale for the mid-term exam. I'm satisfied with the current arrangement for assessment. The course as I teach it divides naturally in two parts, and the mid-term exam is timed to come at the end of part one. I think the exam serves two main purposes. First, it provides an occasion and incentive for students to review and integrate their understanding of the first half of the course. Second, I think it gives a useful reading of where students stand at that point in the semester, in terms of their conceptual understanding of the material.

External Aids Do you allow students to use any external reference aids on exams? (For example, open book, open notes, student supplied note pages or formulas, instructor supplied note pages.)

For instructor supplied aids, please describe. For any student supplied aids, what restrictions do you enforce? (For example, number of pages, content such as not allowing worked out examples, authorship such as personally generated vs. photocopied.)

The mid-term exam is in-class, open book, with no restriction on what notes students can bring. I want the exam to assess a student's conceptual understanding. In the past, my knowing, as I made up the exam, that students would have their books and notes, helped me design questions that called for understanding rather than regurgitation and ritual arithmetic. After a while, my exams settled into a pattern that has remained pretty stable in recent years. The down side of open book exams is that students tend to underestimate the need to study in advance, and that's why, in recent years, I've taken to giving short, weekly closed-book quizzes. Students know very specifically which one or two topics each quiz will cover. I give 10 and drop the lowest 3 grades. The main purpose is to provide an incentive for students not to fall behind, to provide a timely check for students on how well they are doing, and to provide an easy way for students to register their competence. Most students end up with a high quiz average.

Use of Technology Do you allow/encourage students to use any technology on exams? If so, what type (computer, calculator) and what restrictions do you put on their use? Do you include any additional safeguards against cheating based on the technology? Does the technology used on exams correspond to the technology used on homework or in class?

I haven't regarded this as a big issue for exams. Students need the computer for a number of HW assignments, and for their final projects, but I've never given an in-class exam that used a computer, and I doubt that I ever will. I used to be quite consistent in not allowing calculators on exams, which ensured that I designed exams that didn't involve much arithmetic, and that's still my preference. (I can provide output; students have other chances to show that they know how to persuade a computer to give them the output. On the exam, I want to use the time to assess understanding of what the output means.) Not that long ago, I felt that allowing calculators would have given students who had them an unfair advantage, but these days that no longer seems much of an issue.
Constructing Exams In constructing exam questions, do you focus on the content and construct a realistic setting for the questions or do you start with a real problem and try to match the course content? In other words, how much do you strive to use real data as opposed to hypothetical or realistic data in the exam questions?

In constructing the exam, how do you decide how many points a question should be worth? What percentage of the points on the exam would you say are for primarily conceptual knowledge/interpretation vs. calculation/mechanics?

Real data. I always use real data, and I never use data sets that are in the book, or that I've used in class as examples, or that I've ever used on an exam previously. I want every data set to be both real and new to the student. Imposing these constraints on myself means that I spend quite a bit of time, during the weeks leading up to the exam, looking for suitable data sets.

Point values. Questions are grouped by data set. A typical exam might involve 3 or 4 different data sets. Some data sets may be rich enough to support several questions. Others may lend themselves to just one. To help students plan their allocation of effort, I try to create a set of main questions that count equally. (The number varies from exam to exam, but is usually between 6 and 12 main questions.) Some questions may have several parts, but here, also, I try to create divisions into parts that count equally.

Conceptual vs. ritual. For the most part, I rely on HW and weekly quizzes to check for computational ritual. I want the mid-term exam to deal with conceptual understanding. Here I have in mind two main kinds of understanding: interpreting results (going from the abstract numbers or plots to their concrete meaning in relation to the applied context), and recognizing abstract structures (going from the concrete applied scenario to the relevant abstract models or methods). Although that's my main goal, I try to start the exam with a first problem that I hope all prepared students will find straightforward. This one tends to be somewhat more mechanical. Also, unavoidably, most problems have a somewhat more mechanical element to them, and this strikes me as perfectly OK.

Exam Writing Process Describe the PROCESS that you use to write an exam from scratch. For example, how do you get started? What steps do you go through? Do you have another instructor review your questions? How do you decide if the exam is reasonable timewise? Also, after you have given an exam, how do you decide for yourself if it was a good exam?

In the weeks leading up to the exam, I look for data sets. (They must be both real and new, where "new" means not in the book, and not on any previous exams or quizzes ever given here.)
Next, I write questions tied to the data sets, based on what I consider a reasonable analysis given what we've done so far in the course.
Next, I look duplications (I won't ask for the same thing to be done to two different data sets) and omissions. (Sometimes I have to find another data set to make sure that everything gets covered.)
Then I break the questions into equal size parts, to help students plan their time.
Finally, I check for length by taking the exam myself. However, since I try to allow students extra time, I don't worry a lot about getting the length exactly right.

I suppose you could call this a "data driven" approach to creating the exams. Steps 1 and 2 are the important ones; the rest is mainly grooming. I really do allow the data sets to suggest the questions, based on what I consider a reasonable analysis would look like. My hope is that this process helps keep the exams more like what statisticians actually do than would be the case with a different process. If, as I plan the exam, there are gaps in coverage, that tells me I need to find an additional data set of a different sort.

Exam Grading In grading the exam, do you use an analytic or a holistic scoring scheme (assigning points for individual steps or overall "level" of solution)? Do students start at 0 points and earn points for correct statements or do they start at 100% and lose points for mistakes? Do you give partial credit for answers and if so, how do you assign partial credit? Do you have an expected "average" score on the exam for each class? Do you "curve" exam scores?
Do you use a straight total or percentage correct to be added to other points in the course to contribute towards a total grade, or do you give a grade to each exam?

In assigning course grades, do you "curve" student scores? Based on what mechanisms?

I'd never thought explicitly about the analytic/holistic polarity; I guess I use a mixture. A few of my exam questions are straightforward to grade. For example, if I give a linear decomposition of a two-way ANOVA and ask students to fill in an ANOVA table, I'll devise a way to assign points to each entry in the table -- so many points for getting all the df right, for example, with part credit for getting some but not all df correct. This sounds analytic, but the student who gets all but two df right and has them all add up to the correct total has recognized a principle that has escaped the student who gets the same number of df right but doesn't have them add correctly, so the first student gets a higher partial score. This strikes me as more holistic. For other kinds of questions (e.g, I give an ANOVA table and cell means for two-way ANOVA, and ask for an interaction graph and discussion of what the results mean), I start holistic and work toward analytic: I read through a dozen answers, comparing solutions with my own sense of what a complete and correct solution should be. Then I try to devise a way to assign points to parts of solutions that will give the best answers the top scores. Typically this means that it is possible to score more points on the question than the point-value I've assigned to the question as a whole. I don't actually award the extra points, but the system does provide more than one way to get full credit for a solution.

I always assign letter grades, sometimes with a waffle attached (e.g., A-/B+), and I always curve the grades. I explain to students in advance of the exam, and again when I return them, that my exams tend to be hard, and that I think hard exams are important for their learning: the hard exams reflect high standards for what I want them to learn. But I add that giving hard exams means that raw scores don't give a fair indication of how well they've done, and that the scores need to be rescaled to adjust for the level of difficulty of the exam. I adjust the scores by comparing the distribution of raw scores with my sense of which papers should be As, which Bs, etc. Then I create an ad hoc formula that converts As to the 90-100 range, Bs to 80-89, etc. (I tell the formula when I return the exams.)

Preparing Students for the Exam How much class time do you spend reviewing for an exam? Do you offer out-of-class review sessions? Do you supply review sheets and/or review problems? Is student participation in such review experiences mandatory or optional?

I usually hold a one-hour review session in preparation for the exam. Typically, I also hand out a copy of the exam from the last time I taught the course. I post solutions to that exam, and hold an optional session to go over the exam with any students who have questions about it.

Post-Exam Feedback How much class time do you spend going over the exam afterwards? What do you discuss (e.g., the entire exam, the most difficult problems, how the exam was scored)? How specifically do you discuss student responses to exam questions after they have been returned? What information do you convey to students about score distributions? Do you give them access to an exam key? What other types of feedback do you give them about their exam performance? (Do you write comments on the exams, correct their mistakes, or just indicate points? Do you consider students as losing points for mistakes or as gaining points with correct answers?)

I usually spend an hour going over the exam afterwards. I discuss how I graded the exam, and put the grade distribution on the board. (Typically, all grades are C- or better. When I give the distribution, if there's an F or D, I don't include it.) I post a copy of the exam with solutions and the point values I use for giving partial credit. In going over the exam itself in class, I spend most of the time on the questions that the students had the most trouble with.