183 / 2018-03-30 12:22:15

The Development of an English Test to Measure Critical Thinking Ability

Test development,Critical thinking,Logical thinking,Analytical abiliy,Inference

全体主题 > Standards in developing language assessments

In the present study, we have developed a test called the English Critical Thinking Test (ECTT), currently comprised of 20 multiple-choice items, which measures both English proficiency and critical thinking ability. To evaluate the validity of the test, we built a validity argument based on Chapelle, Enright, and Jamieson’s (2008) framework. A total of 81 Japanese first-year university students participated in this study. As for the first inference, namely, domain definition, based on previous studies we defined the target domains: consistency, analysis, and inference. In addition, we set the target English proficiency at level B1 of the CEFR (Council of Europe, 2001). The backing for the warrant of the evaluation inference was obtained from classical item analysis, and we finally excluded six items. In addition, item- and person-fit statistics were examined using Rasch modeling. As a result, two misfit participants (INMS > 1.5) were dropped for the analyses that followed.
With regard to the generalization inference, Cronbach’s α coefficients were relatively lower than expected due to the lack of items and participants. The backing for the warrant of the explanation inference was gathered by conducting factor analysis. Consequently, three expected factors, namely, consistency, analysis, and inference, were extracted. Additionally, the results of a questionnaire asking the participants which abilities they thought they used supplied strong backing for the explanation inference. As the backing for the warrant of the extrapolation inference, a series of Pearson’s correlation analyses were performed among the ECTT, the 11-item Japanese Critical Thinking Test (JCTT), an external criterion test, and the English Proficiency Test (EPT), which consists of 28 items gathered from EIKEN and TOEIC tests. The results showed that there was a correlation trending toward significance between the ECTT and the JCTT (r = .22, p = .054) as well as a significant correlation between the ECTT and the EPT (r = .36, p = .001). These levels of correlation coefficients align with the results of a previous study on different Japanese critical thinking and cognitive ability tests (Hirayama, Tanaka, Kawasaki, Kusumi, 2010). The sixth inference, namely, utilization, was justified by providing the test outcomes of the three subcomponents and elucidating the interpretation of these scores to the participants. Overall, this study found that the ECTT measures both critical thinking ability and English proficiency; further investigation with more participants and items is necessary for the clarification of these results.