?

Log in

Previous Entry | Next Entry

Week 12: Online Assessments

Chapelle-Douglas' article (2006) explores the important developments implied by the uses of technology assessment. It mentions, as an example, TOEFL (Test of English as a Foreign Language) as a test that can be taken, regardless of where ELLs live. As far as assessments is concerned, an issue attracts my curiosity: legitimacy and honesty in answering a test. Since an online test can be taken anywhere (not necessarily, on a supervised testing site), how can we ascertain the legitimacy of the answers? Can the tested students be entrusted with their own answering? Can online tests be almost synonymous with "take-home" tests?

Comments

( 25 comments — Leave a comment )
pkyount82
Apr. 4th, 2009 04:14 pm (UTC)
online assessments
I think online assessments/online take-home tests are appropriate for college level online classes. Part of being a successful student is knowing where to look to find the answers. In an online classroom it is appropriate to have exams online. The college student will get the most out of what he/she puts into the class. If he/she wants to have a friend over to take the test for him/her, then he/she probably won't get as much out of the class than the student who takes the test on his/her own and puts more effort into the class. Now, for high stakes tests such as the TOEFL, GRE, SAT, etc., I believe these tests should be given in supervised testing centers. When someone's acceptance into a college, grad school, international school or career is on the line, I believe the testing process should be as honest and as fair as possible.
jose_estevez28
Apr. 4th, 2009 04:35 pm (UTC)
Re: online assessments
I imagine by your input that an online test would depend on its importance. If it is a self-evaluation test, the student can be entrusted on his/her own, unless he/she wants to do a disservice by self-lying; if you are talking about institutions who issue a test that is set to enforcre a canon of standards, I would say
it requires supervision; standards have to be set to confirm correct or defined knowledge or action. In the world of academics. it is a must. However, if a test involves creativity by implementation (i.e. arts), what kind of evaluation who be used? A rubric, perhaps?
salvaldez
Apr. 7th, 2009 05:22 pm (UTC)
Students should be given the benefit of the doubt, at least in the beginning. There should be some level of trust established between students and the teacher, so as not to automatically assume that there is an element of cheating involved. Strict guidelines should be issued at the beginning of the course to combat dishonest test-taking and enforced when need be. After all, it's the students who are paying for the course, in the end, they will be the ones losing out.

On line tests can almost be synonymous with "take-home" tests, which is not necessarily a bad thing. In my experiences with on line coursework when I was looking up an answer to an on line quiz, I found myself encouraged to continue reading, and most of the time was forced to read more of the book to find all of the answers. Making the questions challenging, so that students don't answer everything correct on the first try (if your offering more than one opportunity) will assure the teacher that the student is actually reading.

How about issuing a follow-up quiz in person after the on line test, to assure that the student actually did the test themselves? A clear set of expectations with dismisal from the course if cheating is involved should be stated at the beginning, sort of like a preemptive notice.
jose_estevez28
Apr. 12th, 2009 01:21 am (UTC)
I agree
I agree. Failing a test can be hard on the pocket. Who knows? It may benefit students to be independent researchers......
loltauruslol
Apr. 8th, 2009 01:47 am (UTC)
response to online assessment
Although online assessment can be completed as long as students have internet access, it still needs to be supervisied in a site in order to maintain the fairness and avoidance for students who really want to cheat. Particularly, example you mentioned in your questions. Students have to go to testing center in order to take the exam. In addition, students also are not allowed to bring any personal belongings during the test. Pens, paper, cel phone ...should be stored in the locker. Although it's still questionging the online assessment, the technolgoy do provides some "adaptive" assesment by using technolgoy, which is the paper-based can't do.

for your second your question: Can online tests be almost synonymous with "take-home" tests? If students are not coming to a site to take test, I would say it's similar to take home exam, because there were able to find other resource to find answer. However, it depends on the purpose of test. It would be to test student and motivate them to learn how to find out answer to answer. This is a benefit.

There is no best answer to decide online test/test good or bad, or effective to test students' level, rather a learner should be assessed in different ways. Sometimes, students are learning "test-taking" strategy" in order to pass the exam. However, they still have hard time using a language.


jose_estevez28
Apr. 12th, 2009 01:26 am (UTC)
Re: response to online assessment
So true; "test-taking strategy". Great part of our professional lives involve that. There might not be a clear answer; it depends on the importance of a test; that is, if the same is designed with academic standards in mind.
alinalac24
Apr. 8th, 2009 04:15 pm (UTC)
Re:Online Assessments
Computerized testing, often described as facile and efficient, has some drawbacks. The most common disadvantages are computer glitches, errors in content, and security lapses. There have been several documented occurrences of security breaches. One example is the Graduate Record Examination, which in 2002 was apparently "hacked" and sold on the internet in Asia. This was a live and active exam, not a practice test. This had damaging consequences, not only for the test administrators, but for the thousands of students taking the test.

On the practical side, some test takers report that it is more difficult to navigate back to rework problems. Some test takers are resistant to the computerized testing process because they are accustomed to taking notes and circling questions and/or answers for later review. Others say that they read more quickly and more easily on paper than on a glaring computer screen. Another incident of standardized test cheating on the GRE occurred when a student on the East coast relayed information to students on the West coast; that scheme took advantage of the three-hour time difference between the two regions. While electronic glitches are rare, they have been known to occur, for instance when computer crashes voided the efforts of thousands of GMAT takers. There are also cases in which the correction software has corrupted scores.

jose_estevez28
Apr. 12th, 2009 01:31 am (UTC)
Re: Online Assessments
Excellent point; it is very hard to monitor technical problems from afar; there is still the advantage of raising a hand and get help. Hackers can render a test useless if they can find the answers. It is remarkable that the old "pencil-and-paper" way has not become a thing of the past.....
maracucha_mama
Apr. 8th, 2009 07:17 pm (UTC)
Online Assessments
I think that most high-stakes tests such as the TOEFL are administered in supervised testing sites, where pre-registration and IDs are required, and the computers used are not connected to the internet at the time of testing. However, as Alina has so ably pointed out, these are not immune to problems. However, it seems to me that the advantages to using computerized assessment in such cases greatly outweighs the disadvantages, and organizations just have to continue to take precautions against security and technology glitches.

As far as using online tests which students take away from the classroom, I agree with Petra that college-level students get out of a class what they put in, and it is in my mind appropriate to trust them with taking their own tests.

When I was in college in the late 70's/ early 80's I took some take-home tests that were also closed-book tests, and we just signed the test booklet with the university's honor code. But honestly, I'm not sure I'd trust college students in this way these days, so I would view online tests as open-book/open notes, etc. But open book tests can be constructed to be quite challenging and are very appropriate in some cases. The test could also be constructed to allow only a certain amount of time on each question or total time, which also would force students to be pretty familiar with the material before taking the test. Also, if the test includes essay questions, these reflect the student's ability to synthesize or apply the info learned, so whether the student is using notes or not is less important.

Online testing in high school would need to be taken in a supervised classroom. If taken from an outside computer they would definitely need to be considered open-book tests and would likely be open to honesty violations (students taking the test for other students). But they could be useful for students to practice with or as homework assignments (which are also a kind of assessment, just usually worth less).
jose_estevez28
Apr. 12th, 2009 01:41 am (UTC)
Re: Online Assessments
This is interesting; a take-home test that can be challenging; for example, take this history test; you can use any sources you wish to substantiate your answers, EXCEPT YOUR CLASS BOOKS. This can certainly make a test a challenge (I know from prior experience; that was EXACTLY what the college professor made us do). I guess everything has to be made clear to the students, even for a "take-home".

Supervised classroom (on-line testing or paper testing) might still be norm for a while, for the sake of the institution.
abbylibby715
Apr. 9th, 2009 03:01 am (UTC)
Online Assessments
I agree too that it depends on the student who is taking the online assessment. An ELL child coudn't take an online assessment. Online assessments should be focused to adult ELLs depending on the setting of the learning enviornment. Also as Sal said teachers should give students the benefit of the doubt as they take online assessments. Also these online assessments shouldn't be only assessment used in a distance ed course either. I like as Valenzuela(2002) wrote to use Multiple Compensatory criteria to assess students. So online assessments should be addition to other assessments in a distance course.
electric_box
Apr. 9th, 2009 04:10 pm (UTC)
These are definitely dilemmas that administrators of distance learning courses struggle with, and such tests are not nearly as high stakes as TOEFL. How can you ensure that students do not cheat or receive external help in the completion of unproctored online final exam for a distance chemistry course. I think the general consensus so far is that there will be a portion of students who will seek outside help and assistance, so that assessments must be designed with this in mind.

As a result, many distance ed courses still require the completion of exams (even via distance) to be proctored in some way - either on-site, on designated sites or computers, or under the affirmed supervision of an adult or educator who must observe the process and attest to the fact that no outside help was received.

Is it possible that Chapelle and Douglas (2006) mean that ELLs can take the TOEFL in any country and not necessarily from their homes?

I do think that certain online tests CAN be synonymous with take-home tests. However, the higher the stakes, the less I think we can trust test-takers and the subsequent results of unproctored online tests. As everyone most likely experienced with the Dialang test, a diagnostic self-assessment (low stakes test), there is little inducement and absolutely no benefit from "cheating" or receiving outside assistance. In the case of the TOEFL, there is potentially a lot to be gained from achieving a higher score by any means possible.

Interestingly, this is not a new problem. In some countries, there are professional test-takers who can be paid to take the TOEFL for you. They are selected based on physical resemblance to the test-taker of record, so they can fool proctors when IDs are checked. I believe they are called ronin. Cracking the TOEFL is a thriving business.
jose_estevez28
Apr. 12th, 2009 01:49 am (UTC)
I didn't know that cheating has become that organized in those countries. I guess that the bottom line is that the means of taking a test, (on-line, paper, "take-home", etc.) would be congruent with its importance. Safety measures would have be taken as seriously as the test.
manmantx
Apr. 9th, 2009 09:47 pm (UTC)
I think that any subject is learned based on the desire or necessity of the learner. A student will not take a placement test online if this was not voluntary. This, of course, if the student is intending to learn the target language. If the student is taking a class online and just taking the test as part of a requirement, then we may encounter some dishonesty or cheating. As I have learned through the master’s degree courses, a second language is learned greatly based on the desire of the learner. For that reason, I believe most language tests are not monitored as a standard test. It is important for teachers and school districts to entrust the students and make them responsible for their grades. In the end, If I don’t read, study, do my homework, and cheat, I may graduate but may be a mediocre professional that will not go far. The same is for students and their school career.
manmantx
Apr. 9th, 2009 09:51 pm (UTC)
I think that any subject is learned based on the desire or necessity of the learner. A student will not take a placement test online if this was not voluntary. This, of course, if the student is intending to learn the target language. If the student is taking a class online and just taking the test as part of a requirement, then we may encounter some dishonesty or cheating. As I have learned through the master’s degree courses, a second language is learned greatly based on the desire of the learner. For that reason, I believe most language tests are not monitored as a standard test. It is important for teachers and school districts to entrust the students and make them responsible for their grades. In the end, If I don’t read, study, do my homework, and cheat, I may graduate but may be a mediocre professional that will not go far. The same is for students and their school career.

This is Lorenzo.
nwl984
Apr. 11th, 2009 03:25 am (UTC)
As someone who has experienced taking a computer-based TOEFL, I have to say that it differs from a take-home test. Since the TOEFL has different sections that should be completed within a limited time frame test taker would not have the opportunity to cheat or ask for help. A take-home test actually gives students the opportunity to use their textbook and other sources to complete the test but when a computer based test has to be done within a certain time students have to focus on anwsering the questions before they are automatically referred to a different section of the exam. I have not seen any computer-based tests that do not have a specific timing. IELTS, GRE, and TOEFL are all tests that should be completed in less than 4 hours and they are all long tests which contain different sub-tests such as reading, writing, or math. So I believe that everyone can rely on the legitimacy and honesty of the answers of the tests takers of the above exams.
hafaadai
Apr. 11th, 2009 08:18 pm (UTC)
response
I think that if the test is high stakes such as TOEFL and other such exams, then they should be administered in a testing site. As far as online tests just to assess a student in say, for example, a college course, then I don't see any problem in giving the students access to the test no matter where they are. I have taken several online courses when I attended community college, and the fact that I didn't have to be physically present was one of the benefits of taking such courses. The instructors have varied ways to assess the students on an online setting. For example, some courses required no exams, but weekly blog-like postings to a topic to ensure the student's participation and to check if the required readings are met. There were also those with timed exams administered only within a specific time period. I think that if the objective is to check for comprehension, then I would require online essay exams or research papers submitted online, but not necessarily multiple choice-type questions which are easier for students to just guess the answers and hope for the best.
clezcano
Apr. 11th, 2009 11:27 pm (UTC)
online assessments
I Think students need to know they can be trusted when taking online assessments because if the individual is serious about his/her goals, he/she won't cheat. I understand that online courses require the students to take their final exam in a site where colleges have surveillance. In reference to other countries and U.S. included, I would think students might tend to find ways to pass the test, but again, it would depend on the individual's honesty.
clezcano
Apr. 11th, 2009 11:39 pm (UTC)
online assessments
I Think students need to know they can be trusted when taking online assessments because if the individual is serious about his/her goals, he/she won't cheat. I understand that online courses require the students to take their final exam in a site where colleges have surveillance. In reference to other countries and U.S. included, I would think students might tend to find ways to pass the test, but again, it would depend on the individual's honesty.
raluca_2009
Apr. 12th, 2009 05:03 am (UTC)
Online assessment has its own advantages and disadvantages, his own supporters and detractors as well. The accuracy and legitimacy of the answers are sometimes placed under question marks, the originality and creativity are challenged. What are the advantages that an electronic/computerized form of assessment brings in addition to a pencil-and-paper test?
Test-makers claims that the scores of computerized and pencil-and-paper tests are equivalent are inadequately supported. In fact, research studies find there usually is a difference. Most studies show higher scores for paper-and-pencil exams, but a few have found advantages for those who take computerized tests. These averages may mask individual variations. Some respondents may still get lower scores even if the average score increases. Also, some types of questions perform differently on the two types of tests.Computerized tests constrain test-takers compared to paper-and-pencil tests. With computerized versions, test-takers are unable to underline text, scratch out eliminated choices and work out math problems -- all commonly-used strategies. Studies also suggest that computer screens take longer to read than printed materials, and that it is more difficult to detect errors on computer screens.Most computerized tests show only one item on the screen at a time, preventing test-takers from easily checking previous items and the pattern of their responses, two other practices known to help test-takers. Scrolling through multiple screens does not allow side-by-side comparisons.Test-takers with the ability to manipulate computer keys rapidly may be favored by long passages that require reading through many screens.Test-makers may try to use computerizd exams to circumvent Truth-in-Testing disclosure requirements. ETS has not revealed how it intends to continue making test questions and answers available to university admissions test-takers.
Computers may worsen test bias. The performance gap which already exists on multiple-choice tests between men and women, ethnic groups, and persons from different socioeconomic backgrounds could widen as a result of computerized testing. Schools with large minority or low-income populations are far less likely to have computers, and poor and minority children are much less likely to have computers at home 2,3. White students are more likely to have earned computer science credit than either African American or Hispanic students.The additional cost of computerized tests is certain to have a large effect on who chooses to take them. Poorer students are unlikely to take the computerized GRE, for example, because it costs nearly twice as much as the paper-and-pencil version.
Thank you for considering my prompt.
Kind regards,
Raluca
raluca_2009
Apr. 12th, 2009 05:05 am (UTC)
I have just posted my prompt, but the time and date are not accurate again. I sent my reply on 11th April at 11:50 pm.
Thank you for considering that.
Raluca
esther_harris
Apr. 13th, 2009 04:01 am (UTC)
I have to say that the issue of honesty will depend on the moral principles the student or learner may have. Ultimately the student will learn if he/she is able to answer the online quizzes or tests properly and as the time passes, if he/she has developed good studying skills and discipline, with the result that he/she will be successful in their future careers.

I think online testing is a great idea and a good interaction with the students. According to Chapelle and Douglas, the teachers want to use online tests with their students and get diagnostic information on the results. I think the course management software is a good way for the students to try these quizzes and tests online. Then the results can be easily submitted to the teachers and students. However, efficiency is important and could be obtained with certain features such as a sensitive response analysis (Chapelle and Douglas, 6).

There are Universities that have designed and offer self-access activity courses where the student can assess his/her own learning. To me it is an easy, less stressful and happy course to take. I believe that some students will be honest in answering the questions but there will always be those who want a short-cut. For the online test developers, technology is not able to detect who is being honest and who is not. I don't think technology will ever figure out how to detect that. All I can say is that it is necessary to give the benefit of the doubt to the student and come up with tests that require a certain amount of time to answer the questions. This will encourage the student to be aware of the necessity of having to study prior to taking the test. The alternative is to have the student take the online test at school under supervision. I believe the take-home test can be difficult and at the same time less stressful.



rkymtnsky
Apr. 13th, 2009 03:33 pm (UTC)
I think online tests should be designed in such a way that it works as a further academic exercise that helps the student gain more knowledge on the subject. However, it would vary greatly depending on the subject and the implication of the test. High stakes tests such as SAT, ACT, GRE, TOEFL and many other professional tests do warrant proctoring but some subjects such as sociology, psychology or other similar classes can use assessment that elicit textual response. In foreign languages and ESL online classes, the teacher should continually check the student’s proficiency through CMC, both synchronous and asynchronous, and have a level of expectation from the student.
I was wondering if the teacher can proctor the test on SKYPE. It would require the teacher having multiple windows. Is this possible?
elisa7410
Apr. 14th, 2009 02:07 am (UTC)
I think that if a student is serious about his/her learning cheating on a test will simply not be an option. When it comes to high stakes tests such as the TOEFL, it is possible that students might find a way to cheat; however, doing that will only affect that individual in the long run. Let's say a students scores really high on the TOEFL, what's going to happen when they arrive at a foreign and realize that they cannot pass their classes because their English is not good enough? That's just money down the drain. Students might be able to cheat on a test, put they cannot cheat their whole four years of college.
loula_1983
Apr. 15th, 2009 07:38 pm (UTC)
maria's response
Jose,
You raise a very difficult question. Yes, I think students are more tempted to cheat on an online test or to work in groups, but that doesn't mean that they will. Some ways to acertain the legitimacy of these tests is to make the questions open ended rather than multiple choice. However, these types of tests are more difficult to grade and there is no imediate feedback. This is a tough issue for me. I would rather avoid online tests or if they are necessary I would not weigh so much of the students' grade on them.
( 25 comments — Leave a comment )

Profile

callspot
The CALL Spot

Latest Month

May 2009
S M T W T F S
     12
3456789
10111213141516
17181920212223
24252627282930
31