Study on the Applicability of the Self-assessment Scale for Pragmatic Competence of College English Learners

doi:10.15936/j.cnki.1008-3758.2024.02.015

Abstract

Abstract: The evaluation of pragmatic competence is important for promoting pragmatic teaching and learning， but the existing evaluation tools are not rich enough. Therefore， it is significant to proactively explore the applicability of self-assessment scales for pragmatic competence. A questionnaire survey and test were conducted for 483 higher education students ranging from specialists to postgraduates， 439 valid data were obtained， and Rasch model was used to carry out a specific study in terms of validity validation and item function bias. The results showed that the self-assessment scale has good reliability， validity and discriminability， and its 28 descriptors do not show significant bias in terms of language proficiency. However， there are several descriptors with significant bias in terms of gender and major. Therefore， it was suggested that the bias of certain items should be reconsidered in the future application of the self-assessment scale for pragmatic competence to ensure its fairness.

Key words: pragmatic competence; self-assessment scale; applicability; validity; differential item functioning

CLC Number:

H030

PENG Yuchen. Study on the Applicability of the Self-assessment Scale for Pragmatic Competence of College English Learners[J]. Journal of Northeastern University(Social Science), 2024, 26(2): 135-143.

References

[1]Rose K R. Pragmatics in teacher education for nonnative-speaking teachers:a consciousness-raising approach［J］. Language， Culture and Curriculum， 1997，10(2):125-138.
[2]Taguchi N. Pragmatic comprehension in Japanese as a foreign language［J］. Modern Language Journal， 2008，92(4):558-576.
[3]刘建达. 中国英语能力等级量表与英语教学［J］. 外语界， 2019(3):7-14.
[4]任伟，李思萦. 二语语用习得国际研究热点及趋势［J］. 外语教学， 2018，39(4):19-23.
[5]李清华，邹润. 二语语用测试研究30年［J］. 解放军外国语学院学报， 2015，38(4):43-50.
[6]姜占好，周保国. 学习者语用能力评估研究［J］. 外语教学， 2012，33(5):45-48.
[7]Hudson T，Detmer E，Brown J D. A framework for testing cross-cultural pragmatics: technical report 2［R/OL］. Honolulu: University of Hawai'i， 1992.
[8]Hudson T，Detmer E，Brown J D. Developing prototypic measures of cross-cultural pragmatics: technical report 7［R/OL］. Honolulu: University of Hawai'i， 1995.
[9]Tada M. Assessment of EFL pragmatic production and perception using video prompts［D］. Philadelphia: Temple University， 2005.
[10]Yamashita S O. Six measures of JSL pragmatics: technical report 14［R/OL］. Honolulu: University of Hawai'i， 1996.
[11]Roever C. Testing ESL pragmatics: development and validation of a web-based assessment battery［M］. Frankfurt: Peter Lang， 2005.
[12]Roever C. Validation of a web-based test of ESL pragmalinguistics［J］. Language Testing， 2006，23(2):229-256.
[13]Itomitsu M. Developing a test of pragmatics of Japanese as a foreign language［D］. Columbus: The Ohio State University， 2009.
[14]Roever C，Fraser C，Elder C. Testing ESL sociopragmatics: development and validation of a web-based test battery［M］. Frankfurt: Peter Lang， 2014.
[15]Taguchi N，Roever C. Second language pragmatics［M］. Oxford: Oxford University Press， 2017.
[16]Liu J. Developing a pragmatics test for Chinese EFL learners［J］. Language Testing， 2007，24(3): 391-415.
[17]刘建达. 话语填充测试方法的多层面Rasch模型分析［J］. 现代外语， 2005(2):157-169.
[18]刘建达. 中国学生英语语用能力的测试［J］. 外语教学与研究， 2006(4):259-265.
[19]刘建达. 语用能力测试的评卷对比研究［J］. 现代外语， 2007(4):395-404.
[20]韩宝成，黄永亮. 中国英语能力等级量表的研制——语用能力的界定与描述［J］. 现代外语， 2018，41(1):91-100.
[21]黄玮莹，刘建达. 中国英语语用能力等级量表内容效度研究——以表达说话意图子量表为例［J］. 解放军外国语学院学报， 2022，45(4):10-15.
[22]陈新仁，杨昆. 英语口语测试中的语用能力测试题设计研究——以《中国英语能力等级量表》中五级语用能力描述语为例［J］. 浙江外国语学院学报， 2021(4):1-8.
[23]马颖，陈旭远. 基于《中国英语能力等级量表》建构大学英语语用能力评价模型的实证研究［J］. 东北师大学报(哲学社会科学版)， 2021(2):151-164.
[24]李民. 《中国英语能力等级量表》语用能力量表效度研究——以写作有关描述语为例［J］. 浙江外国语学院学报， 2021(4):9-18.
[25]孙莉，伏潇涵. 基于多面Rasch模型的语用能力量表自评效度研究［J］. 西安外国语大学学报， 2021，29(2):52-57.
[26]吴莎. 国际英语考试对接中国英语能力等级量表的预期效应［J］. 现代外语， 2019，42(5):672-683.
[27]朱正才，李俊敏. 《中国英语能力等级量表》描述语偏差研究［J］. 现代外语， 2021，44(1):113-122.
[28]Roever C. DIF in the assessment of second language pragmatics［J］. Language Assessment Quarterly， 2007，4(2):165-189.
[29]张文霞，郭茜，吴莎，等. 我国英语教学现状与改革建议——基于外语能力测评现状及需求调查［J］. 中国外语， 2017，14(6):18-26.
[30]Rasch G. Probabilistic model for some intelligence and achievement tests［M］. Chicago: University of Chicago Press， 1980.［31］Bond T G，Fox C M. Applying the rasch model: fundamental measurement in the human sciences［M］. New York: Routledge， 2015.［32］AERA，APA，NCME. Standards for educational and psychological testing: national council on measurement in education［R/OL］. Washington D.C.: American Educational Research Association， 2014.［33］Wright B D. Reliability and separation［J］. Rasch Measurement Transactions， 1996，9(4):472.［34］Jr Fisher W P. Reliability， separation， strata statistics［J］. Rasch Measurement Transactions， 1992，6(3):238.［35］朱莹莹. “95后”大学生道歉言语行为的性别差异探究［J］. 现代交际， 2021(13):139-141.［36］周薇，周民权. 国内社会性别语言定型研究现状及问题分析［J］. 外语教学， 2017，38(4):46-50.