When I look at the technology assessment tools I look at:
1. Does it meet the need? Am I using the tool for the right reason. An example of this is multiple guess testing. It is easy to grade (automatic) and it has it's place. However, if I am trying to gauge critical thought I am not sure this is the right tool and the instructor may just need to do a little more work and grade from a rubric.
2. Will it work on all platforms? More and more students are using iPads, Laptops, Smart phones and other such mobil technology. We need to make sure that the assessment tools work on all such devices unless we are setting the standards of the course so that it excludes non-working devices (and that could have cost implications and retention implications on the students).
3. Do the students respond well to it? I think the thought I am trying to share here is, "will the student take more time to figure out what the technology is (or how to use it) than answer the question?" I have seen an instructor add drag and drop matching to a quiz only to realize that the majority of the students on the iPads are unable to use drag and drop questions. Sort of an "oops...." moment.
-Chris
Jeremy,
What do you mean by assessment quality?
Dr. Kelly Wilkinson
I believe the 3 most important things are ease of use, assessment quality, and efficiency.
Thomas,
I agree with you completely. Don't you wonder when the printing press made textbooks available, did they ponder the role of "technology" of printing? The learning should drive the use!
Dr. Kelly Wilkinson
Melissa,
Great point. You don't want technology to go from tool to barrier quickly. Students need to be able to focus on the learning not focus on the technology. Technology should be ubiquitous. (I love that word).
Dr. Kelly Wilkinson
Melissa,
You are so right. Your comment regarding accuracy is so important as well as reliability. We have state testing online and the computers each institution has varies greatly. This can really be an issue in high ed online ed.
Dr. Kelly Wilkinson
This is a great question and one that I reflect on from time to time. I have never been a fan of technology for technology sake. First, you must consider the objectives that are being measured, and can they be effective measured given the assessment being considered. Second, user comfort level and access. The assessment must be in a format that the students are comfortable in using, and be based in a technology that is widely available. Many students are not willing or are unable to download software. Additionally, it the assessment is not in a format the student is comfortable with, they may not be able to complete the assessment at all. Lastly, there must be technical and instructional support. The instructor must be able to provide front line assistance and support with technical support available when needed.
Michael, these are great!
I find the "ease of use" is truly essential within an online learning environment. Our students face daily challenges and time constraints. We do not want to add any more frustrating challenges to their daily responsibilities. We simply want our students to have an enjoyable learning experience. Also, technical support is another important aspect that should be handy for our students.
In assessing the use of technology assessment tools in an online learning environment, the three most important things to consider are accuracy, reliability and student feedback. It is essential to continuously review and update the technological supporting course materials. For example, in our college we are required to do a QA course assessment of all the course modules and assignments. I find this to be helpful so that we can possibly find any glitches or content issues before the start of the course. Technology can be dependable and unpredictable. It is important as instructors to remain active within our course and technology assessments. It needs to be accurate and reliable. In assessing early on, we can possibly prevent any further challenges our students may face throughout their course learning experience.
The best technology assessment tool is to get our student's feedback as they are the key people who experience the online course experience directly. They can provide helpful feedback, challenges or concerns throughout their course experience.
Charles,
What a great post! Alignment is so important. It may not seem to be important, but students will notice if there isn't alignment.
Dr. Kelly Wilkinson
Jim,
That is a great way to collect data regarding technology. That will be so helpful to determine resources needed.
Dr. Kelly Wilkinson
Chander,
You are right. You also want to make sure that you get the data you need to tell the story of the student and the course.
Dr. Kelly Wilkinson
I really feel that the alignment of the material being tested and the course objectives is the most important item with online testing. Secondly the alignment of the testing format with the teaching format is important and finally the understanding of the technology and its purpose by the students and teacher is crucial. The alignment and the understanding must be strong in order to get true assessments of the learning and not just getting assessment of the technology. Sometimes I feel that successful students who take online assessments are those who understand the process not necessarily understand the learned materials. The assessment needs to assess learning not use of technology.
thank you
Charles Vakos
Hi Kelly,
To answer your question, I always ask my students about their current experience with the software that I'm teaching them. Based on their responses, I can more accurately diagnostically assess the.
I can further consider the Formative assessment by reviewing how they are doing as the course progresses through reviews, quizzes, etc.
The 3rd tool I would use is the Summative Assessment method of assessment by seeing what they know at the conclusion of a project which would give me a more accurate way of measuring their progression.
Jim deLapine
Important considerations for the selection of technology in an online learning environment include i) appropriateness of the tool for specific assessments, ii) familiarity with the tool to make sure students can access the assessment and receive proper feedback and iii) suitability of the tool for assessing students performance based on the content of the course. Thanks
amanda,
These are good criteria. I don't know now "non-traditional" online education is now.
Dr. Kelly Wilkinson
Students understanding of the material that is presented in a non-traditional environment, knowing the level of engagement the student is putting forth and making sure that the student understands the objectives (goals) of learning the materials.
Michael,
You bring up important points. I know some institutions who have had so much problems with LMS that it became the barrier to learning. Connection is everything. I also had teachers taking an online course and couldn't do their work at school because of the firewalls. You do have to be aware of the connectivity.
Dr. Kelly Wilkinson
A great question and three things I consider important when it comes to the use of technology in the online learning environment are:
1. Availability/Connectivity - What I mean by this is how easy is it to access the bandwidth that the students, staff, the administration and faculty use? To keep it fair, I am discussing the actual educational institutions (bandwidth, spectrum, cloud or other [new] aspect of technology service offered to remain connected.
2. Technological Support - This is a lifeline to all things technological, in the event something does go awry then being able to call someone who can not only talk you through issues (if you have any with gaining access to the school site-servers etc.) but sometimes even small personal things/settings they can talk you through to ensure you get and / or remain connected.
3. Ease of use - Regardless of the technology, if on a mobile devices, a specific LMS or platform, it is always best to keep the process of the technology and the use of it as simple as possible. If there are many steps involved with gaining access to a site, submitting answers and reviewing or viewing information, as this could possible deters students from participating and become rather frustrating for faculty.
Though there are many areas to consider, I think these are foundational things which need to be considered when using technology.
Mike
Kelly,
Love your post. Reliability of the application is so important. You make such an important point regarding the output of results. You have to be able to measure the learning and report the results. Very astute observations.
Dr. Kelly Wilkinson
