Looking at the unit, course, program as a whole summative evaluation would judge the worthiness or effectiveness. This type of evaluation states or reports the reason intervention was successful to whomever the stakeholder may be (McArdle, 2007). Included in a summative evaluation there is generally four reasons or indicators of success to be reported. Originally developed by Donald Kirkpatrick in 1959 but made popular by later publications, the indicators for successful training are reaction, learning, behavior and results (Kirkpatrick, 1996). Reaction describes how well the learner liked their process of learning. Reaction is generally what the learner thought and felt about the training and can be learned through the use of tools such as surveys. Learning is the extent of knowledge gained or shown learning which can be proven by testing or grading. The behavior portion describes how the learners were able to use their training or new knowledge obtained from the learning experience. Were they able to implement or apply their new knowledge in the desired way? Results show what value or improvement the training had to the company such as cost reduction or increased production. In order to achieve purposeful evaluation it is recommended the all of Kirkpatrick’s indicator of success are used fully.
With Kirkpatrick’s indicators of success in mind summative evaluations are focused on outcome. In order to measure outcome this type of evaluation uses objective findings. It uses methods such as questionnaires, surveys, interviews, observations, and testing for data collection in order to produce its report. A good balance of both qualitative and quantitative methods will produce a full picture of whether or not the instructional designer’s intervention was successful (McArdle, 2007). In order to truly validate whether or not an intervention was successful, the summative evaluation must be able to determine if a project or training has met its objectives and goals. A good evaluation shows the tracking of the impact of any changes to performance. Summative evaluation should also compare the results to any other previous projects and give the information needed to improve future projects or trainings.
Kirkpatrick, D. (1996). Great Ideas Revisited. Techniques for Evaluating Training Programs. Revisiting Kirkpatrick's Four-Level Model. Training And Development, 50(1), 54-59.
McArdle, G. E., (2007) Training, Design, and Delivery, ASTD, VA
jacob ,
Glad the student feedback has been beneficial to you. Hope they are enjoying the change up in the learning process. Thanks again for your input.
At my instituion, we are just starting up with blended courses where the students meet on ground once a week and have an online portion of the course once a week. The student feedback has been a great benefit for me as an instructor as i develop and improve these courses.
Philip,
Yes, the formative feedback during the course can be very helpful. If you take their feedback from the formative assessments and use it in the current class, they will see that you want their feedback and will provide more constructive feedback at the end of the course. Thanks!
francis,
We do want to know if the students are learning, but also we need to know how we can improve as online instructors. Thanks for your input.
I think student feedback is the tool that would give the most comprehensive feedback. Unfortunately, the way it is normally done is at the end of a course when students are just glad it is over with and many don't give constructive feedback. I think the day-to-day informal feedback is most valuable and best intended to improve the course.
I would need an evaluation that allowed the student to give me opinions on my efforts plus the books and the individual projects. It would be necessary to know if the students feel they are learning and expanding their knowledge.
John,
Self-assessment is a good way to review things, but don't dismiss the power of peer and student evaluations as well. They bring a whole new perspective to everything. Thanks!
Jerry,
Five week course are very intensive for sure. The first week can become very important. I also send an email to my students a week before the course starts and ask them to go into the LMS and look around and get familiar with everything because we are going to hit the ground running! That helps.
I appreciate self-assessment most, as I am an EXTREMELY tough critic, and I tend to find areas of improvement even where other methods of assessment do not.
John D. Wise, PhD
Dr. Crews,
Thank you for the insight. Most of the classes that I teach are intensives and only 5 weeks long, so I try to start feeling my way around the first week with the hope that if I learn anything substantial I can at least use it for my next set of students. As you already know, my next sets of students my comprise of entirely different sets of learners with different mindsets...
I am still looking for ways to motivate my students the first week of class to spur them into taking more interest in the evaluation and feedback process.
The challenge continues...
Jerry.
Jerry,
Good point. I think others feel the same way. What I try to do is ask students on a formative basis (about 3 weeks into a 15 week course and again about 6 weeks) as to what they would like for me to stop doing, start doing and continue to do (Stop, Start, Continue method). They typically make suggestions and if I can change something based on their suggestion, I do. If I can't change it, I explain why and that usually is because it's tied to a learning objective. But, then the students see you want their feedback and you do things based on their feedback. I then stress that their input is critical to making this course better. Just a thought.
Thanks again for your input.
Hello Dr. Crews,
I use a combination of different types of evaluation depending on the student population. The most challenging for me has been student's participation. I teach an elective course, so many of my students do not feel "vested" in the course content. If I can get more students to contribute to the learning community I think we will be able to get a better picture of the effectiveness of course, and help us determine if we are meeting the overall course objective.
Jerry.
Heather,
The students provide excellent feedback as they are the ones submerged into the content and the course. I make many changes in my courses based on student feedback. Thanks!
I think the most helpful feedback is the student's. What they thought about the program and if they were learning in the program. If they aren't learning anything from it, then there is no point in the program. They are the ones telling other's about it and are the ones directly effected by it.
Daniel,
It sounds like you provide a variety of opportunities to assess your students' knowledge. When we use a variety of techniques, we can then adjust our teaching if needed. Thanks for your input.
I believe a combination of the weekly discussion board responses, the homework, and the quizzes combine to give me a flow of the comprehension for each student. This gives me a sense of the dificiencies in the way the material is absorbed by the class as a whole and gets me thinking of better delivery methods for understanding of particular concepts.
Heather,
Professionals in the field can give a whole different dimension to the evaluation process. It's another work related process that the students can benefit from. Thanks!
Lucille,
A variety of evaluations may be used to fully understand what the students know, and possibly more important, what they do not know. It is definitely an ongoing process. Thanks!
Although it is something that we already do, as a new instructor to my specific course, I am looking forward to my students' portfolios being reviewed by professionals in the field.
