国内外六款AI大语言模型英语写作文本特征对比研究
基于人工智能(AI)的大语言模型正在全方位重塑语言教育生态。大语言模型通过基于海量文本数据的预训练,能够捕捉到语言的复杂性,生成适应不同语境与主题的文本,并能够为学习者提供即时反馈、写作文本生成和逻辑框架,有望改变传统的二语写作教学模式。由于每种大语言模型在训练方式、底层架构和算法方面并不完全相同,因此在写作文本质量上也可能有差异。本文通过实验研究,基于Coh-Metrix文本分析工具,对ChatGPT、Gemini、Claude、DeepSeek、文心一言和Kimi等主流大语言模型在英语同题写作文本的词汇多样性、句子衔接性、段落衔接性、句法复杂度和文本可读性进行了对比分析,研究结果可为英语二语写作教学中的模型选择提供适配度参考。
| [1] |
MINDNER L,SCHLIPPE T,SCHAAFF K.Classification of Human- and AI-Generated Texts:Investigating Features for ChatGPT[M].International Conference on Artificial Intelligence in Education Technology. Singapore:Springer Nature Singapore,2023:152-170. |
| [2] |
苏祺,杨佳野.语言智能的演进及其在新文科中的应用探析[J].中国外语,2023,(3):4-11. |
| [3] |
BUBECK S, CHANDRASEKARAN V, ELDAN R, et al. Sparks of Artificial General Intelligence: Early Experiments with GPT-4 [EB/OL]. (2023-3-22) [2023-4-13]. https://arxiv.org/abs/2303.12712. |
| [4] |
郭茜,冯瑞玲,华远方.ChatGPT在英语学术论文写作与教学中的应用及潜在问题[J].外语电化教学,2023(2):18-23+107. |
| [5] |
吴琼.汉语二语者、母语者及ChatGPT生成记叙文写作质量和词汇复杂度对比研究[J].世界汉语教学,2024(4):517-532. |
| [6] |
HERBOLD S,HAUTLI-JANISZ A,HEUER U, et al.AI,write an essay for me:A large-scale comparison of human-written versus ChatGPT-generated essays[EB/OL]. (2023-4-22). https://arxiv.org/abs/2304.14276. |
| [7] |
SMITH G, FUNK J.When It Comes to Critical Thinking,AI Flunks the Test [EB/OL]. (2024-3-13). https://www.chronicle.com/article/when-it-comes-to-critical-thinking-ai-flunks-the-test. |
| [8] |
GRAESSER A C,MCNAMARA D S,CAI Z,CONLEY M,LI H,PENNEBAKER J.Coh-Metrix measures text characteristics at multiple levels of language and discourse[J].The Elementary School Journal,2014,115 (2):210 - 229. |
| [9] |
MCNAMARA D S.Coh-Metrix:Capturing linguistic features of cohesion [J].Discourse Processes,2010(47):292-330. |
| [10] |
CROSSLEY S A,MCNAMARA D S.Predicting second language writing proficiency:The roles of cohesion and linguistic sophistication[J].Journal of Research in Reading,2012,35 (2):115–135. |
| [11] |
ULLMANN T D. Automated analysis of reflection in writing: validating machine learning approaches[J].International Journal of Artificial Intelligence in Education,2019,29 (2):217-257. |
| [12] |
SANTOS T. Professors’ reactions to the academic writing of nonnative-speaking students[J].TESOL Quarterly,1988,22(1):69-90. |
| [13] |
ASTIKA G G. Analytical assessments of foreign students’ writing[J].RELC Journal,1993,24(1):61-70. |
| [14] |
NATION P.Learning vocabulary in another language[M].Cambridge:Cambridge University Press,2001. |
| [15] |
KYLE C,CROSSLEY S A.The relationship between lexical sophistication and independent- and source-based writing[J].Journal of Second Language Writing,2016,34:12-24. |
| [16] |
CROWHURST M. Syntactic complexity and teachers’ quality ratings of narrations and arguments[J].Research in the Teaching of English,1980 (14):223-231. |
| [17] |
BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[M]//LAROCHELLE H,RANZATO M,HADSELL R,et al.Advances in Neural Information Processing Systems 33 (NeurIPS 2020).Red Hook,NY:Curran Associates Inc.,2020:1877-1901. |
| [18] |
YANG K,KLEIN D,PENG N,TIAN Y.Improving long story coherence with detailed outline control[M]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Toronto:Association for Computational Linguistics,2023:3378-3465. |
/
| 〈 |
|
〉 |