I support the use of pre- and post-tests to evaluate whether the objectives of the course have been met. Since they are used more as an evaluation measurement and not just to provide a “score†for the student, they can be adjusted and manipulated to focus on the objectives that have been set. Not only are they beneficial to the instructor to be able to determine whether the course has been successful in increasing the students’ knowledge base in the topics, they are also beneficial to the students so that they can also see for themselves. Sometimes students are hard on themselves (and their facilitators) and feel as if they haven’t really learned anything. But if they are able to see that their percentages have increased at the end of the timeframe, it could be a source of encouragement for their future courses.
If we are to go back to the benefits of the pre- and post-tests for the facilitators, it allows them to evaluate whether their students are learning or not. If the scores increase at the desired rate, the facilitator will know that they are doing well in the structure and presentation of their course. However, if the scores are stagnant this will alert the facilitator that changes are need. Stagnant scores should motivate them to go back and evaluate their course to see where changes/enhancements can be made.
Thomas,
It's never easy. ;-) You are right in that a variety of evaluations provides a variety of information.
Asking students for feedback or asking a colleague for feedback will result in different perspectives.
You also touch on qualitative vs. quantitative data and you are correct. You may need to get facts and numbers in different ways.
Thanks!
I do not intend to take the easy way out on this response but I can not say truthfully that there is a single type of evaluation that will provide the data necessary to close the loop. Each of the evaluations brings a different perspective on the course, its’ relevance, and the effectiveness of the teacher and material. Only through a combination of evaluation types will you get a comprehensive view of the course.
Student evaluations through structured graded surveys provide a narrow view of their satisfaction with the course, material, instructor, and possible campus. Open interviews will allow a more thorough analysis to be performed as those being interviewed are encouraged to expand on their responses. Peer reviews allow us to develop and integrate best practices. All of these are only part of the overall continues process of closing the loop, each plays a part to add depth to the analysis process and help effectively close the loop.
George,
I agree that evaluations should come from several sources. As a colleague may review your course for content or instructional strategies, your students may review the course design or let you know what is not easily accessed or understood.
It's a never ending process. Keep asking for feedback and revise the course content, layout, etc. Thanks for your input.
Tena
1. My own thoughts on how the course progressed at the end of each term.
2. The student evaluations with relevant comments. As a graduate student, I did not get many constructive comments on my instruction good or bad. I have gotten more comments lately as I have made a request to the students to make relevant comments on the evaluations.
3. Paying attention to the outcome of assignments. Length of time students used to perform the assignments to gauge the amount of time a test on the material may take for example.
4. The institution I work for has started a peer evaluation program that I hope to learn from in two ways. I learn from watching my peers and get constructive feedback on how to become better from them as they observe me.
The most comprehensive course evaluation is multiple source evaluations. I call these a reality check to confirm or deny my own personal evaluation of the course. These evaluations include those from students, peers, and the Lead Faculty or Dean. It is important to keep the course objectives at the forefront in the evaluation process and to look at how the objectives were actually met. (A “running†evaluation is grades and assignments quality, but evaluations might shed some light on why these turned out the way they did.) In any evaluation from the students and other stakeholders I would include best and worst practices (open ended questions) so that I could see how my methods meet the needs of the students and how other professional view my teaching.
Nancy,
You are on target! Thanks for your input. It is important to evaluate your online course in a variety of ways. You are right. The main outcome is to improve the course, communication, navigation, etc.
It's also important to remember that you don't just ask for feedback at the end of a course, but throughout the course. Nice job.
Thank you.
I believe all methods of evaluation are important. When a course is developed, it should begin with the objectives and the content should be mapped to those objectives. I like to develop my assignments and tests based on the learning objectives and then structure the content to ensure students have learned what they are being tested on, which is linked back to the objectives. So, reviewing course mapping is important.
Peer evaluation is also very important as "two sets of eyes" are always better than one. There is always something someone else will be able to see that will improve upon the course.
Feedback from learners is extremely important. Evaluations are a great tool, but being in tune with the students and listening to common questions/comments and clarifying content where students seem to get "tripped up" are also very effective informal methods of feedback that continually help to improve the course.
In addition,trends in specific test items also help to identify problem areas in the content. If all students are repeatedly getting the same questions wrong, the content needs to be reviewed and revised for clarity.
Bill,
Nice job. Thanks again for all of your input. Feedback from peers and students is a great combination. Having peers join in from the student perspective is an excellent way to get a feel from both aspects.
Thanks!
The most comprehensive feedback that would allow one to close the loop would be provided by the evaluation type of peer observation/review of course materials. The person has total access to the course. The review can be asynchronous or synchronous, and be as thorough as desired. Some schools conduct partial peer reviews, from a student’s perspective, by having reviewers pose as alias students.
- Dr. Bill Lembke
George,
Thanks you for providing input to several of these forums. I know others will benefit from your input.
You are right in that feedback should help students improve. Assessment should not be something we do to our students, but with our students. When you get the opportunity, include self- and peer evaluation as well.
As you noted, individualized assessment is important. And, the point about varying your feedback is great too. "That was good!" ;-)
Thank you.
I try to set feedback expectations with my evaluations and feedback to help them improve. When I begin any class I add information about what the students can expect from me concerning feedback. For example, I explain to the class that I will review all of the exams the day after they are due. I will take questions concerning the solutions from the exam at that time, and not before. I will post their grades in the gradebook at such and such time.
I try to comment on every assignment turned in for grading. To accomplish that I like to use grading templates. I have a chart per assignment with phrases I have developed that can be easily individualized. This allows me to vary the feedback instead of always stating "that was good!" I now can write you did an exceptional job on your ... (thesis statement, transitions, conclusions, etc.)
Joseph,
The students do provide great feedback so that instructors can improve the course the next time it is taught. It's also helpful to ask for that feedback throughout the course and not just at the end of the course.
Thanks for your input.
Jennifer,
Students do provide much needed feedback to the instructor. You should ask for formative feedback throughout the course and summative feedback at the end of the course. However, if you have the optin of asking another instructor to review the course, this will also provide good feedback for course revision and content delivery.
Nice job.
I have to agree that the most comprehensive would come from actual students. The students are the ones with a vested interest in the course and have tons to gain. Their feedback will be or should be carried with great result.
In reviewing the different types of evaluation, I believe the most comprehensive would come from actual students who take your course. The problem sometimes, as already stated by a few, is that students don’t want to take the time to complete a survey or course evaluation and look at it as “one more thing to do.†If you are a fair and open-minded instructor it would be nice to ask your students throughout the course what they thought. It could be as simple as a casual discussion such as, “what did you like or dislike about this project, assignment, etc.†Having your students take part in the course evaluation throughout the quarter/semester could provide feedback and might also empower as well.
George,
Tell me more about the Quality Assurance Specialist evaluation. That sounds interesting.
I like your term 360 degrees evaluation. We must, as instructors in an online environment, look at all angles to make the teaching and learning better.
Thank you.
Dr. Crews,
I will use the summative evaluation that will include studentvevaluation, self-evalaution, Quality Assurance Specialist evaluation.
I consider all these 360 degrees evaluation.
Hearty cheers,
Dr. George Ojie-Ahamiojie
Clayton,
Yes, the Start, Stop, Continue method is very effective in gettting feedback from students so that projects, assignments or the way you are conducting class can be altered for everyone's beenfit.
You are right taht you may not be able to do everything the students want, but you must address the issues and explain why you cannot do it if this arises.
Thanks.
Donna,
AWESOME! You are right on target. Yes, evaluation should be done on a continual basis. Just like learning, it's a lifelong process.
Excellent point.
