Tests as a Technique for an ESP Coursebook Evaluation
Keywords:
ESP coursebook evaluation, design-based research, test specification, design principlesAbstract
Design-based research evolved near the beginning of the 21st century as a practical research methodology that could effectively bridge the gap between research and practice in formal education, as it aims at both developing theories about domain-specific learning and the means designed to support that learning. Since ESP teachers often design their own learning materials, this kind of research enables them not only to evaluate their quality and gradually improve them but also to produce design principles. Accordingly, the aim of my ESP coursebook design-based research is to evaluate the quality of a coursebook pilot version I developed for the students from the Faculty of Information Technology at Brno University of Technology in the Czech Republic, and consequently formulate necessary modifications based on design principles. I first frame the basic concept of design-based research and ESP testing. Then, I describe the methodology of the ESP coursebook evaluation. Results indicate the requirements for the coursebook redesign, which involve adding more tasks for the acquisition of linguistic means for expressing different language functions, including more material for recycling and reinforcement focused mainly on vocabulary practice and increasing the level of difficulty of listening passages. Based on these results, I discuss the ESP coursebook design principles related to multi-skill tasks, and professional vocabulary and language functions acquisition.
References
Alderson, J. C. (1988). Testing and its administration in ESP. In D. Chamberlain & R. J. Baumgardner (Eds.), ESP in the classroom: Practice and evaluation (pp. 87–97). Oxford: The British Council.
Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington D.C.: American Educational Research Association.
Bachman, L. F. (2004). Statistical analyses for language assessment. Cambridge: Cambridge University Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press
Bakker, A., & Van Eerde, H. A. A. (2013). An introduction to design-based research with an example from statistics education. In A. Bikner-Ahsbahs, C. Knipping, & N. Presmeg (Eds.), Doing qualitative research: methodology and methods in mathematics education (s. 429–466). New York: Springer. Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14.
Barnard, R., & Zemach, D. (2014). Materials for specific purposes. In B. Tomlinson (Ed.), Developing materials for language teaching (pp. 306–323). London: Bloomsbury.
Baumann, J., Blachowicz, C., Bates, A., Cieply, C., Manyak, P., Peterson, H., & Graves, M. (2013). The development of a comprehensive vocabulary instruction program for nine- to eleven-year-old children using a design experiment approach. In T. Plomp & N. Nieveen (Eds.), Educational design research. Part B: Illustrative cases (pp. 23–47). Enschede: SLO – Netherlands Institute for Curriculum Development.
Chráska, M. (1999). Didaktické testy. Brno: Paido.
Chráska, M. (2007). Metody pedagogického výzkumu. Základy kvantitativního výzkumu. Praha: Grada.
Council of Europe. (2011). Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Council of Europe.
Danaye, T. M., & Haghigi, S. (2014). Evaluation of ESP textbooks: Evidence from ESP textbook of computer engineering major. International Journal of Research Studies in Language Learning, 3(2), 55–68.
Davies, A. (2001). The logic of testing languages for specific purposes. Language Testing, 18(2), 133-147.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8.
Douglas, D. (2000). Assessing languages for specific purposes. Cambridge: Cambridge University Press.
Douglas, D. (2013). ESP and assessment. In B. Paltridge & S. Starfield (Eds.), The handbook of English for specific purposes (pp. 365–393). Chichester: John Wiley & Sons, Inc.
Dudley-Evans, T., & St John, M. J. (1998). Developments in English for specific purposes. Cambridge: Cambridge University Press.
Ellederová, E. (2016). English for information technology. Brno: VUT FEKT.
Ellederová, E. (2018). An ESP coursebook evaluation: Teachers’ and students’ viewpoints. In V. Janíková & S. Hanušová (Eds.), Research in foreign language pedagogy (pp. 149–165). Brno: Masarykova univerzita.
Ellederová, E. (2020). Konstrukční výzkum učebnice pro výuku angličtiny pro specifické účely: Hodnocení pilotní verze učebnice. Pedagogika, 70(1), 69–96.
Field, A. (2013). Discovering statistics using IBM SPSS Statistics. London: Sage.
Habtoor, H. A. (2012). English for specific purpose textbook in EFL milieu: An instructor’s perspective evaluation. International Journal of Linguistics, 4(3), 44–59.
Halliday, M. A. K. (2004). The language of science. New York: Continuum.
Hendl, J. (2009). Přehled statistických metod. Analýza a metaanalýza dat. Praha: Portál.
Hinkel, E. (2006). Current perspectives on teaching the four skills. TESOL Quarterly, 40(1), 109–131.
Hulstijn, J. H., & Laufer, B. (2001). Some empirical evidence for the involvement load hypothesis in vocabulary acquisition. Language Learning, 51(3), 539–558. Hymes, D. (1974). Foundations of sociolinguistics: An ethnographic approach. Philadelphia: University of Pennsylvania.
Jackson, S. L. (2009). Research methods and statistics. A critical thinking approach. Belmont: Wadsworth Cengage Learning.
Jacoby, S., & McNamara, T. F. (1999). Locating competence. English for Specific Purposes, 18(3), 213–241.
Jafarabadi, M. N. S., Zarghi, N., Zolfaghari, V, & Kargozari, M. R. (2014). The effect of washback on reading comprehension of medical students in English for specific purposes classes. Future of Medical Education Journal, 4(4), 28–31.
Jebahi, K. (2009). Using a commercially developed ESP textbook: A classroom dilemma. The Asian ESP Journal, 5(2), 75–92.
Lightbown, P. M., & Spada, N. (2006). How languages are learned. Oxford: Oxford University Press.
Mishan, F., & Timmis, I. (2015). Materials development for TESOL. Edinburgh: Edinburgh University Press Ltd.
Mol, H., & Tin T. B. (2008). EAP materials in New Zealand and Australia. In B. Tomlinson (Ed.), English language learning materials. A critical review (pp. 74–92). London: Continuum.
Nation, I. S. P. (2000). Learning vocabulary in another language. Cambridge: Cambridge University Press.
Pearson, I. (1988). Tests as levers for change (or ‘putting first things first). In D. Chamberlain & R. J. Baumgardner (Eds.), ESP in the classroom: Practice and evaluation (pp. 98–107). Oxford: The British Council.
Plomp, T., & Nieveen, N. (Eds.). (2013). Educational design research. Part A: An Introduction. Enschede: SLO – Netherlands Institute for Curriculum Development.
Razmjoo, S. A., & Raissi, R. (2010). Evaluation of SAMT ESP textbooks for the students of medical sciences. Asian ESP Journal, 6(2), 107–149.
Reeves, T. C., & Amiel, T. (2008). Design-based research and educational technology: Rethinking technology and the research agenda. Educational Technology & Society, 11(4), 29-40. Rinder, J., Geslin, T. S., & Tual, D. (2016). A framework for language and communication in the CDIO syllabus. In J. Rinder (Ed.), Proceedings of the 12th International CDIO Conference (pp. 17–32). Turku: Turku University of Applied Sciences. Retrieved from http://www.cdio.org/files/document/cdio2016/72/72_Paper_PDF.pdf
Scio. (2014). Teorie a metodika testů. Retrieved from https://www.scio.cz/o-vzdelavani/teorie-a-metodika-testu/
Trna, J. (2011). Konstrukční výzkum v přírodovědných praktikách. Scientia in Educatione, 2(1), 3-14.
Van der Akker, J. (Ed.). (2006). Educational design research. Abingdon: Routledge. Vičič, P. (2011). Preparing materials for ESP teaching. Inter Alia 2, 107–120.
Weir, C. J. (2005). Language testing and validation. An evidence-based approach. Basingstoke: Palgrave Macmillan.
Zangani, E. (2009). The ESP textbook problem: The evaluation of ESP textbooks in humanities in the undergraduate program of Iranian universities. The Asian ESP Journal, 5(2), 93–106.
Downloads
Published
Issue
Section
License
Copyright (c) 2022 International Journal of English for Specific Purposes
This work is licensed under a Creative Commons Attribution 4.0 International License.
