Use of Exams Are "exams" a component of your students' course grade? What percentage of the overall grade are from exams? What is the format of your main exams? Would you characterize these exams as mostly multiple choice, short answer, longer answer questions, or some mixture?
Are these in-class exams or take-home exams?
What time restrictions do you place on the exams?
How many students do you expect to finish in the time allotted?
Why do you use exams? Do you feel exams are an important reflection of students' abilities?
Do you take any safeguards to try to minimize student cheating? Please describe the concern and how you respond (e.g., seating arrangements, multiple versions of exams).
I usually give two midterm exams and one comprehensive final. Exams are typically 15% of the overall course grade with the final sometimes weighted more heavily. The questions are a mixture of short-response and longer questions, with a mixture of computation (often with access to a computer) as well as interpretation and explanation. The midterms are given in class, usually for 50 or 70 minutes. I would like students to have sufficient time to finish, but this is very difficult to manage with this time requirement and some will complain about the time constraint. When I previously taught on the semester system, a component of the final exam was take home, providing students with more time and access to their notes. (There were numerous versions of the take home component, students were given different data sets and asked 2-3 data analysis and inference questions about the dataset.) I don't usually give multiple versions of in-class exams except if I have multiple sections on different days. With the amount of writing and explanation students are required to do, I hope that copying of numerical answers is not advantageous. Other assessment components in the course allow more time and more collaboration. My goal on exams is to see if the students can apply their knowledge, fairly quickly, and demonstrate individual understanding of important concepts.
External Aids Do you allow students to use any external reference aids on exams? (For example, open book, open notes, student supplied note pages or formulas, instructor supplied note pages.)
For instructor supplied aids, please describe. For any student supplied aids, what restrictions do you enforce? (For example, number of pages, content such as not allowing worked out examples, authorship such as personally generated vs. photocopied.)
I typically provide students with a set of formulas. I have also experimented with open-book exams and notes pages. I'm not sure I see a strong advantage with any of these approaches right now. My main concern is that students realize that they are not responsible for memorizing formulas, but instead need to be concerned with applying the formulas, choosing the correct formula in a particular situation, and interpreting and explaining their results.
Use of Technology
Do you allow/encourage students to use any technology on exams? If so, what type (computer, calculator) and what restrictions do you put on their use? Do you include any additional safeguards against cheating based on the technology? Does the technology used on exams correspond to the technology used on homework or in class?
Students are always allowed/encouraged to use a calculator and often a computer as well. We do sometimes safeguard that they are not accessing the internet during an exam. The technology required on exams directly parallels what they have already done during class and on assignments. Students are also encouraged to ask me Minitab questions during the exam, as I am not testing their Minitab knowledge as much as encouraging them to use the technology to expedite their analysis of the data.
Constructing Exams
In constructing exam questions, do you focus on the content and construct a realistic setting for the questions or do you start with a real problem and try to match the course content? In other words, how much do you strive to use real data as opposed to hypothetical or realistic data in the exam questions?
In constructing the exam, how do you decide how many points a question should be worth? What percentage of the points on the exam would you say are for primarily conceptual knowledge/interpretation vs. calculation/mechanics?
Usually I start by thinking about which topics I want to cover. I have given students a detailed review sheet and I use that as a guide of topics to choose from. I also spend some time looking through test banks and previous exams. I'm always searching for interesting, current contexts and continue to use genuine data as much as possible. Once I have the context/scenario, I continually refine the statistical content that I want to assess. I try to make sure the points given to a problem are proportionate to the amount of time it should take them to complete. I hope at least half the points are conceptual over mechanical. I also try to make sure that even when there are multiple parts to a question, later parts can still be answered even if earlier ones cannot. I hope this reduces the number of distinct contexts students need to read through to help with the time constraints.
Exam Writing Process
Describe the PROCESS that you use to write an exam from scratch. For example, how do you get started? What steps do you go through? Do you
have another instructor review your questions? How do you decide if the exam is reasonable timewise? Also, after you have given an exam, how do
you decide for yourself if it was a good exam?
I start by reviewing the content I want to cover and look through other question sources to get the juices flowing (see answer to constructing exams as well). I try to start a few days ahead of time so I have time to put it aside and then look at it again with a fresh perspective the next day. Often I will look through the week's news to find an interesting context. Once I have a tentative exam, I do show it to another faculty member to check for coverage and level of difficulty. I also advocate showing it to a TA to check for reasonableness and clarity. I'm fortunate to have a colleague who will review my exam after each of many revisions. I also time how long it takes me to take the exam (reading the questions and writing complete solutions) and multiply by 5 or 6 to see if that is close to the time allotment the students will have or if I should reduce the number of questions. I decide if the exam is reasonable by considering the point distribution and I even try to think a bit about the useful of each question (how much did I learn about the students). As I am grading the exam I make notes to myself on ways to change the question/grading the next time around.
Exam Grading
In grading the exam, do you use an analytic or a holistic scoring scheme (assigning points for individual steps or overall "level" of solution)? Do students start at 0 points and earn points for correct statements or do they start at 100% and lose points for mistakes? Do you give partial credit for answers and if so, how do you assign partial credit? Do you have an expected "average" score on the exam for each class? Do you "curve" exam scores?
Do you use a straight total or percentage correct to be added to other points in the course to contribute towards a total grade, or do you give a grade to each exam?
In assigning course grades, do you "curve" student scores? Based on what mechanisms?
My scoring system is a mixture of analytic and holistic. Most of the problems are graded fairly analytically as I decide how many points certain errors deduct. On some problems I am more holistic as I categorize the answers into different point levels. On larger problems, the points are broken down into separate pieces, though I am careful to allow students to "carry-through" an error and not mark them off repeatedly for what is really just one mistake. More conceptual problems are often 3 or 4 points. Students typically receive half the points with a reasonable but incorrect attempt and then say 3 of 4 points if they have some correct understanding but have missed a key aspect of the problem.
When I am done, I hope for an average of around 80 percentage points. With the amount of partial credit I give, if a student answers every question incorrectly, that is 50%. Students need to leave several questions blank to score below 50%. If the average is around .80, I do not do any curving or adjusting. If it is a little lower, I focus on the overall course average being around .80. Rarely does the average score fall below that, but then in that case I might make some adjustments when it comes to assigning the final course letter grades (instead of adjusting the individual assessment components). When the average is around 80%, then I assign the letter grades following the 50/60/70/80/90 breakdown, including plus and minus grades.
Preparing Students
for the Exam How much class time do you spend reviewing for an exam? Do you offer out-of-class review sessions? Do you supply review sheets and/or review problems? Is student participation in such review experiences mandatory or optional?
I try to devote up to one class period to answer student questions, though I really try to make sure they bring in the questions instead of me telling them new information (though I'm not always successful). This usually works better during the optional night-before-review sessions I offer. I also give students a set of review problems and post the solutions on-line. Student review questions often center on these review problems. I also give students a review handout. I've recently restructured these to try to be more prescriptive. Instead of "know this and this," I know say, "you should be able to …" and focus more on the skills they need (including what they should be able to explain) and not just mentioning the topic. I should also note that students also take weekly quizzes, typically two before each exam, that I design to include questions similar to what they will see on the exam. This way, students realize that many of the questions will involve explanation and not only (if any at all) calculation. The grading guidelines and feedback on these quizzes should give them information about my expectations for their answers on the exams. Students also have access to model quiz solutions on-line. I also have an online set of multiple choice questions that they have the option of taking before the final.
Post-Exam Feedback
How much class time do you spend going over the exam afterwards? What do you discuss (e.g., the entire exam, the most difficult problems, how the exam was scored)? How specifically do you discuss student responses to exam questions after they have been returned? What information do you convey to students about score distributions? Do you give them access to an exam key? What other types of feedback do you give them about their exam performance? (Do you write comments on the exams, correct their mistakes, or just indicate points? Do you consider students as losing points for mistakes or as gaining points with correct answers?)
I try to find time to go over the exam, or at least very common student errors, in class. I wish there was more time for this. Often I will post a full set of solutions on line. I do try to give students extensive written feedback on their midterms but I'm not sure how much they look at these comments. I often also prepare a handout discussing the most common student errors. My hope is that they will review these concepts as they continue to learn the material and study for the final. I try to be positive and remind them that this is a quarter-long process. Those who did well shouldn't get over confident as the material builds, and those who didn't do well shouldn't get discouraged as they will see many of the same ideas again in the course and will have the opportunity to improve their understanding (and to demonstrate that improvement).
I often show students the distribution of scores or at least the mean/median and max. I have stopped allowing students to resubmit answers for more points but sometimes give very similar questions on the final.