260 / 2018-04-15 23:41:24

Investigating the Accuracy and Usefulness of an Automated Writing Evaluation System for Formative Assessment of L2 Writing

automated writing evaluation,accuracy,usefulness

全体主题 > Standards in validating language assessments

Recently, there is a growing number of studies on the use of tools for automated writing evaluation in writing classrooms, indicating increasing interest in their potential for formative assessment. As an online writing evaluation system, Pigai was used in a college-level English as a Second Language (L2) writing course. The current study examined the accuracy and usefulness of Pigai for formative assessment of L2 writing course.

86 students from an academic writing course were invited to participate in the study. This study addresses the following two research questions: 1) How accurate is Pigai feedback in terms of the errors it can identify in students' writing? 2) How well can students use Pigai feedback to correct errors that have been accurately identified by the system? Quantitative and qualitative data were collected to answer the two research questions. Results showed that accuracy varied across error types. Meanwhile, students demonstrated different abilities to use Pigai feedback to correct their written errors. The findings can inform discussion of whether and how to integrate the use of online writing evaluation system into writing classroom. It also gives implications about the evaluation and utilisation inferences in Chapelle, Cotos, and Lee's (2015) argument.

References
Chapelle, C. A., Cotos, E., & Lee, J. Y. (2015). Validity arguments for diagnostic assessment using automated writing evaluation. Language Testing, 32, 385–405. doi:10.1177/0265532214565386