Dr Wilkinson,
Technology acceptance model (TAM) posits that ease of use and usefulness of technology are critical factors in new technology acceptance.
Austin
Susan,
There are several tools that track clicks and websites that students visit while attending class.
Austin
In considering technology assessment tools, I would consider feedback mechanism, ability to track and make changes and paper format retention.
Austin
Using technology assessment tools can be a great asset to any classroom, however, there are still many things to consider in order to determine if they are appropriate. First, what is the student's compentency regarding technology? Is he/she familiar with navigating the assessment? Can he/she use a computer to the level needed to successfully complete the assessment?
Secondly, one should consider the length of time required to complete the assessment. Some students may require more time than others, so it's important to set appropriate time limits and lengths.
Lastly, you have to consider what you are hoping the student gains out of the experience. I feel the best assessments are those that go outside of what a textbook explains and actually applies the knowledge to an actual situation or event. With online assessments, it can be easy and tempting to open a book and look up answers, however, if you can design things so that there is more effort/thinking involved, then you will get a true test of what the student has learned.
I believe that the tools are only as good as the development.
So as instructors, we need to understand what we are assessing. If the assessment is not measuring the right information, the data is bad.
We need to ensure that our students understand how to access the tools. If they are confused or lack understanding about the assessment tools, the learning curve can be steep.
The tools should never be the end all - they are just a tool to assist and can never replace the human needed to work with the students.
The three most important things to consider when assessing an assessment tool are the type of activity being assessed, consistency of grading and difficulty/ease of technology for instructor/student. For example, the type of assessment I would choose would be different for various activates (i.e. reflective journal, hands on experience (simulations) and general knowledge etc.).
Greetings, Kenya!
I want to respond to the third criterion - will the tool enhance the instructor feedback (fb).
At our university, we read and assess papers and discussion board postings. The assessment tool is written fb, posted with the grade. The university has established some standards for that fb, which is wise, and if those standards are met, the student gets valid, useful fb.
In this situation, the more relevant question is, "What an the instructor do to enhance her use of the tool?" In my practice, I use two strategies. First, I give comments that focus on the CONTENT of the assignment, AND comments that focus on the learning and performing PROCESSES the student used. I feel that the process fb is the more important of the two.
The second enhancement I make to this assessment tool is to write my comments in MS Word, then transfer them to a spreadsheet, then to the gradebook. I use Word because I can use AutoCorrect to program my feedback comments for both content and prcess issues that I see repetitively in students' papers. For example, if I type aparef in Words, that string gets converted to
APA errors in references (deduct x points). Check proper formatting for referencing at http://owl.english.purdue.edu/owl/resource/560/05/ . For referencing electronic sources, see http://owl.english.purdue.edu/owl/resource/560/10/.
I want to respond to the first point,
"1) Is the technology assessment tool being used the best choice for the subject matter?"
"Best choice" is a complex criterion, and I would think it will be rare that any one tool will meet all the requirements we might desire of it. Among the attributes we might want to optimize are cost, complexity, functionality, and appropriateness to the learning objective. March & Simon (1956) said that most managers do not follow a rational decision making model; instead, they satisfice - they find the first choice that meets their needs and go with it.
Given the complexity of the choices we are faced with in this particular choice, and also given the rapidly changing technological environment, we are pretty much forced to satisfice in our choices of tool, then must be alert to opportunities to improve on its function.
My perspective is grounded in the context of teaching for a school that does not afford the instructor the opportunity to choose most assessment tools. Such assessment tools as I am allowed to select are add-ons to the structure already established, such as use of insert comments and autocorrect to generate feedback comments.
First consideration is whether the instructor knows how to use the tool properly. Training should be provided to ensure this.
Second consideration is whether the tool is generating knowledge at a level on Bloom's taxonomy that is appropriate to the learning objectives. The most significant determinants of this are beyond the puview of the instructor, as the tool has already been selected in the course design process. However, the instructor can ensure that the feedback being generated is at a level appropriate to the learning objective. this requires active reflection on the learning objectives while in the assessment process.
Third consideration is whether the student is responding to the information of the assessment. The instructor can generate brilliant feedback, but if the student does not incorporate it into the learning process it is all for naught. For this reason, I refer back to feedback on previous assignment, with particular focus on process feedback, to assess whether the student did this; if not, then I point this out.
the first aspect top consider is whether the assessment accurately reflect the learning objectives of the course or section being facilitated. It can be discouraging to a student to take an assessment that has nothing to do with their readings, studies or discussions during the course and will not reflect their performance.
The next thing is to insure that the assessment is a type that will demonstrate what the instructor wishes it to do, especially when in an online environment. It can be too easy to use multiple choice questions that the computer can grade automatically but if the facilitator wants to see how well someone understands the topic and can apply it then a performance based assessment may be needed.
Finally,the assessment needs to be what the student can use effectively. An evaluation of the assessment by students can help the facilitator to determine if the assessment was workable for the student and that the facilitator could work as well.
I do not teach online but I do use assessment tools in my classroom. I use online testing and the one concern that I always have is the site user-friendly. Can the student use the username and password that I have assigned them to access the test with ease? Another item that I look for -- can the program scramble the questions so students cannot look on another student's monitor and copy the answer. Most of the online assessments sites that I have used have the ability to rearrange the questions and the answers. For reference I give multiple choice questions for my comprehensive finals. I want the student to know what grade they received on their final immediately.
To sum up my criteria for online assessment tools I look for a user friendly environment; the accessibility of the site -- does the student have to download additional software in order to use it; and is the tool the right tool for the activity that I will have the students doing.
Gary,
You make great points. The technology should NOT drive the assessment; you need to determine what is the best way to assess learning and then find the tool that best suits the activity.
Dr. Kelly Wilkinson
Jessica,
What about your needs as you develop an assessment plan for the course?
Dr. Kelly Wilkinson
Thomas,
What a great answer. You need to determine how you want to assess learning then find the tool that works for you.
Dr. Kelly Wilkinson
Yehia ,
You also need to determine what assessment you want to do then find the tool that fits it. I think we get in trouble when we force the tool to do what it really can't.
Dr. Kelly Wilkinson
Nate,
I really like your last statement. We must use a variety of technology for assessment but they must work and be worthy of usage.
Dr. Kelly Wilkinson
Marie,
I agree with you. I wonder if we do a disservice to students not having a frank discussion about what it takes to be successful regarding availability of technology.
Dr. Kelly Wilkinson
Tammy,
Good point. We have to also keep the complaints in check with students. Students will always have to learn new technology throughout their careers. This is a great way to develop that skill.
Dr. Kelly Wilkinson
Emry,
You make a great point. it is incumbent on us as educators to make sure we KNOW how to use the technology. We will lose our on "cred" if we don't make sure we know what we are doing.
Dr. Kelly Wilkinson
Ruby,
Where does your needs as an instructor come into play?
Dr. Kelly Wilkinson
