TOEFL iBT Iranian Test-Takers’ Oral Language Performance: A Comparison between Independent and Integrated Speaking Tasks
Keywords:
TOEFL speaking, Independent task, Integrated task, Complexity, Accuracy, FluencyAbstract
The present study investigated the speaking module of TOEFL iBT with an emphasis on the dichotomy of independent and integrated tasks. The potential differences between the two speaking conditions were intended to be explored based on the oral performance elicited from a group of Iranian test takers. To collect the required data, a simulated version of TOEFL iBT characterized with all test rubrics as practiced by ETS was applied. A total number of 224 oral samples were transcribed and then quantified in terms of complexity and fluency by some internet-based programs including Coh-Metrix 3 and Syllable Counter. Moreover, the accuracy of each oral performance was manually computed for comparison purposes. The results revealed that the two speaking contexts were significantly different in terms of linguistic accuracy as well as two complexity measures representing latent semantic analysis and the mean number of modifiers per noun phrase. However, the two speech-eliciting contexts did not trigger any significant difference in the participants’ oral language performance in terms of fluency and two other complexity measures, namely lexical diversity and left-embeddedness. The findings may have some implications for all TOEFL iBT stakeholders concerning the speaking subtest.
References
Ariamanesh, A. A., Barati, H., & Youhanaee, M. (2020). TOEFL iBT integrated speaking tasks: A comparison of test-takers’ performance in terms of complexity, accuracy, and fluency. Iranian Journal of Applied Linguistics (IJAL), 23(2), 33-62.
Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. A. (2019). Introduction to research in education (10th ed.). Cengage Learning, Inc.
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford University Press.
Barkaoui, K., Brooks, L., Swain, M., & Lapkin, S. (2013). Test-takers’ strategic behaviors in independent and integrated speaking tasks. Applied Linguistics, 34(3), 304-324. https://doi:10.1093/applin/ams046
Biber, D., & Gray, B. (2013). Discourse characteristics of writing and speaking task types on the TOEFL iBT test: A lexico-grammatical analysis. TOEFL iBT Research Report 19. Educational Testing Service. https://doi:10.1002/j.2333-8504.2013.tb02311.x
Brown, A., Iwashita, N., & McNamara, T. (2005). An examination of rater orientations and test taker performance on English-for-Academic-Purposes speaking tasks (TOEFL Monograph Series MS-29). Princeton, NJ: Educational Testing Service.
Brown, H. D. (2004). Language assessment: Principles and classroom practices. Pearson Education, Inc.
Carr, N. T. (2011). Designing and analyzing language tests. Oxford University Press.
De Jong, N. H. (2018). Fluency in second language testing: Insights from different disciplines. Language Assessment Quarterly, 15(3), 237-254. https://doi.org/10.1080/15434303. 2018.1477780
Educational Testing Service. (2020). TOEFL iBT®: Test and score data summary 2019. https://www.ets.org/toefl
Ellis, R. (2008). The study of second language acquisition (2nd ed.). Oxford University Press.
Ellis, R. (2009). The differential effects of three types of task planning on the fluency, complexity, and accuracy in L2 oral production. Applied Linguistics, 30(4), 474-509. https://doi:10.1093/applin/amp042
Ellis, R., & Barkhuizen, G. (2005). Analysing learner language. Oxford University Press.
Frost, K., Elder, C., & Wigglesworth, G. (2011). Investigating the validity of an integrated listening-speaking task: A discourse-based analysis of test takers’ oral performances. Language Testing, 29(3), 345-369. https://doi:10.1177/0265532211424479
Frost, K., Wigglesworth, G., & Clothier, J. (2020). Relationships between comprehension, strategic behaviors and content-related aspects of test performances in integrated speaking tasks. Language Assessment Quarterly. https://doi:10.1080/15434303.2020. 1835918
Fulcher, G. (2003). Testing second language speaking. Pearson Education Limited.
Fulcher, G. (2010). Practical language testing. Hodder Education.
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36, 193-202. https://doi.org/10.3758/BF03195564
Huang, H. T. D., Hung, S. T. A., & Plakans, L. (2018). Topical knowledge in L2 speaking assessment: Comparing independent and integrated speaking test tasks. Language Testing 35(1), 27-49. https://doi.org/10.1177/0265532216677106
Kyle, K., Crossley, S. A., & McNamara, D. S. (2015). Construct validity in TOEFL iBT speaking tasks: Insights from natural language processing. Language Testing, 1-21. https://doi.org/10.1177/0265532215587391
Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
Luoma, S. (2004). Assessing Speaking. Cambridge University Press.
McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied natural language processing. In P. M. MaCarthy & C. Boonthum (Eds.), Applied natural language processing and content analysis: Identification, investigation, and resolution (pp. 188-205). Hershey, PA: IGI Global. https://doi.org/ 10.4018/978-1-60960-741-8.ch011
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press.
Pallant, J. (2016). SPSS survival manual (6th ed.). Open University Press.
Roever, C., & Ikeda, N. (2021). What scores from monologic speaking tests can(not) tell us about interactional competence. Language Testing, 39(1), 7-29. https://doi.org/ 10.1177/02655322211003332
Shohamy, E., Or, L. G., & May, S. (2017). Language testing and assessment (3rd ed.). Springer International Publishing AG.
Skehan, P. (2014). Processing perspectives on task performance. John Benjamins Publishing Company.
Weir, C. J. (1990). Communicative language testing. Hemel Hempstead: Prentice Hall.
Weir, C. J. (1993). Understanding and developing language tests. Hemel Hempstead: Prentice Hall.
Weir, C. J. (2005). Language testing and validation: An evidence-based approach. Basingstoke: Palgrave Macmillan.
Wigglesworth, G., & Frost, K. (2017). Task and performance-based assessment, in Shohamy et al. (eds), 2017. Springer International Publishing AG. https://doi.org/10.1007/978-3-319-02261-1_8
Yan, X., Kim, H. R., & Kim, J. Y. (2020). Dimensionality of speech fluency: Examining the relationships among complexity, accuracy, and fluency (CAF) features of speaking performances on the Aptis test. Language Testing, 38(4), 485-510. https://doi.org/ 10.1177/0265532220951508
Downloads
Published
Issue
Section
License
Copyright (c) 2022 International TESOL Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
