228 / 2018-04-14 10:27:51

Validity standards in language testing research and language in validity standards

validity,Standards for Educational and Psychological Testing,text mining

全体主题 > Standards in validating language assessments

Validity is the cornerstone of test development and use, as exemplified by beginning test development with intended consequences in the AUA (Bachman & Palmer, 2010) and the interpretive argument that accompanies test development in the IUA (Kane, 2006). The Standards for Educational and Psychological Testing (APA, AERA, & NCME, 2014), as the primary professional guidelines of assessment particularly in the US, apply to language testing as to other testing domains. Validity is also considered fundamental within the Standards as it is the first foundational chapter. We will present our investigation of the following research questions: (1) to what extent do recent studies published in language testing journals adhere to mainstream validity theory and utilize the sources of validity evidence presented in the Standards? And (2) how is language as a target construct treated within the Standards, if it is addressed at all? To address RQ1, we present updated results from a survey of publications in three language testing journals evaluating the use of citations from validity theory (Messick, Kane, Bachman & Palmer) and validity standards (the Standards). To address RQ2, we present new results from a text mining analysis in R of the validity chapter in the Standards, investigating the discussion of language as a construct within the Standards. We conclude that many studies claiming to address the topic of validity do not cite foundational sources in validity theory or standards of validation, and studies that are grounded in validity theory and standards tend to overlook certain types of validity evidence. Additionally, we assert that the Standards were not designed with language testing in mind, and additional considerations may be needed when language is the target construct. Our findings have implications for the use of general testing standards in language testing and the future orientation of validation research.