I agree. Many instructors are not very technologically savvy although they may be experts in the subject matter taught.
I think that you are right on target. Subject matter must be considered first.
I like your comment about user friendliness. I agree that this is important.
1. Audience - I must know my audience to decide what form of assessment is the most suitable.
2. Course content - I must consider the subject matter and what type of assessment is the most suitable.
3. Technology available - some assessment methods require technology that may not be available to all students.
1. Availibility of technology. Students may not have the required Internet speed of ability download plug ins.
2. Suitability of the tool for the content of the class. Some content is better tested in the form of essays than tests, for example.
3. Student workload required. Some types of assessments are very time consuming and students already spend a lot of time in their weekly assignments.
The first thing I believe to be important is that the instructor is capable of using the tool. The instructor must be efficient and effective in using the chosen tool. One of my favorite tools is "Track Changes". I find this tool to be very helpful when grading essays and research papers. I also believe that it helps if students have some knowledge of the assessment tools used. My reasoning behind this is that students can sometimes give feedback as to how the tools can be better utilized. Thirdly, a important thing to consider is to use a tool such as WebQuests, which would allow interaction between students.
Earl,
Great points!!!! You used the word efficacy, that is such a great word to describe that the technology should do. You don't want the technology to be the barrier.
Dr. Kelly Wilkinson
The 3 most important things for me to consider are(in order of degree):
1. Efficacy. Whether the tool accurately assesses the content mastered
2. Feedback. Am I able to actually provide substantive feedback on what I've assigned. (Sometimes I tend to have lofty expectations of my ability to read/grade essays)
3. Difficulty. To what extent will the technology itself be an obstacle for students to overcome. (Am I testing their knowledge of the content or their knowledge of how to use a PC?)
Jacqueline,
Great post. You make a great point, about efficient and effective. That is important for the students and the instructor.
Dr. Kelly Wilkinson
Leigh,
That is great. It is nice that you are able to give feedback regarding the effectiveness of assessments. You are right, the tool should be easy to use for both student and instructor.
Dr. Kelly Wilkinson
The first thing I believe to be important is that the instructor is capable of using the tool. The instructor must be efficient and effective in using the chosen tool. One of my favorite tools is "Track Changes". I find this tool to be very helpful when grading essays and research papers. I also believe that it helps if students have some knowledge of the assessment tools used. My reasoning behind this is that students can sometimes give feedback as to how the tools can be better utilized. Thirdly, a important thing to consider is to use a tool such as WebQuests, which would allow interaction between students.
In the environment in which I teach, the assessment tools that I can use are prescribed for me. However, faculty can influence this through the course assessments, and ultimately course development. I would have to say that usability and functionality are two of the top conditions to consider. My students come from a variety of backgrounds; some have very limited computing skills, while others are very advanced. Therefor, any tool we use must be exceedingly simple and clear, and not require advanced skills. The function of any tool must be clear and the "results" provided to students must also be very clear. I would also have to say that the functionality for the instructor should be considered. These tools only work as well as their administrators (the faculty). For this reason, they need to be easy to use, and not take extra time to implement.
Martin,
Great post. Your third point is very important. You want to make sure that the technology is transparent and they only anxiety students exhibit is about the test, not about the technology.
Dr. Kelly Wilkinson
1] Is the technology driving my curriculum or is my curriculum driving the choice of technology?
2] Is my assessment truly measuring what I think it is?
3] Is the technology that I choose to employ to assess my students fair, accessible, and "transparent". By the last, I mean am I grading content instead of a student's comfort level with the technology?
Wenda,
Great point. I actually make practice tests available to students so they can become used to taking exams online. It is different.
Dr. Kelly Wilkinson
Michael,
Great point! adaptive testing is really hard to find in many technology tools. Even the LMS's don't do that well. We have to be able to do that and it can cause frustration on the student's part and increased workload for instructors.
Dr. Kelly Wilkinson
Wenda,
You make a great point about "room to grow". Many times we buy technology that it does one thing well but doesn't do anything else. It can be so frustrating.
Dr. Kelly Wilkinson
The most important thing to consider when assessing the use of technology assessment tools in an online learning environment is first asking the right questions. Is this assessment tool beneficial to the learning process for students and faculity? Ofen times new technology assessment tools could enhance training if implemented at the proper time and input is received from the stake holders of the organization. Reading and understanding the expectation is equally important. Certainly if it is not explained and/or misunderstood, it could cause a major issue. It is just as important to take your time to write carfully. Try to take your time and fully explain your complete thoughts. If this is done properly and completely, your message will be delivered as your intended.
Thanks, Wenda Killings.
One of the first and more advanced features of online assessment tools I would look for is adaptive testing. Many technology certification tests use this technique to test a candidate with questions that get progressively harder or easier in response to the previous questions. So a candidate can pass a test with 10 progressively difficult questions if they are all answered correctly. The second feature would be an extensive question bank that gets updated frequently to reduce the use of brain-dumps to pass a test. The third feature would be simulations portions of the test that allow the candidate to interact with the technology they are being tested on.
1) Does this show me what the student is learning?
- Sometimes tests and quizzes can show what a student has memorized, but a good assessment tool should help the student understand and retain the information
2) Ease of use
- Is the tool easy to use? If not, what instructions can I provide to help students and if there a technical support number available for the tool (if students have questions or run into problems)
3) Will this allow for improvement overtime?
- Is there room to grow if I use this tool? If I use tests/quizzes, can improving the questions each course help the students learn? Will this tool help the student in this class but in other courses (and in their everyday life; like helping them study or retain other information)
Lastly, as I read other posts, I think it will be helpful to see other things to consider when assessing an assessment tool.
