I tend to follow the following outline.
My goal in an exam is to test the material that I told the students was important. I do not try to test material that I did not spend a considerable portion of time going over. This can be challenging when one gives open note/open book exams. Students often do not study enough and are not prepared for the exams. I try to warn them that with only 65 minutes for the exam, they have to be prepared and organized.
I find that I have to make the key BEFORE I make the final copies to distribute. I always find my mistakes in wording or I find the data did not test the concept I wanted, or any such kind of problem. I have difficulty making myself do this, but whenever I don't, I end up announcing several corrections while the students are taking the test and this is not good. This also helps me assess how long students might take to complete the exam.
Time on exams is difficult for me. Students often complain that my exams are too long. My feeling tends to be if they knew the material, it would not take them as long. This takes experience.
I also try to write one question that is a little bit of a stretch of understanding to separate the A's and the B's. My grades tend to be very U-shaped - lots of A's and B's or D's and F's, and very few C's. I want to make sure that the A students really understand the material well.
I typically do not have a colleague look at my exams. I find it difficult to help colleagues when they ask me to look at an exam, so I don't generally ask for their help. The colleague does not know my students, nor what I covered in class.
My guiding principle is basically, if a student comes to class and does the practice homework, will he or she do well on this exam?? If the answer is the student needs additional information somehow to get a B or A, then I think the exam is too difficult.