Enhancing Visual Learning with Keyword Captioned Video
An Eye-Tracking Study
DOI:
https://doi.org/10.1558/cj.20284Keywords:
captioned video, cognitive load, cognitive theory of multimedia learning, eye tracking, keyword captions, listening recall, split-attention effect, visual recall, vocabulary gainAbstract
This current study examined the effects of full captions (n = 37), keyword captions (n = 38), and no captions (n = 37) on students’ ability to gain pictorial knowledge from a 5 minutes and 52 seconds French language video. After completing a pre-vocabulary recognition test to determine the level of equality between groups, students watched the video and immediately completed a post-visual recognition test and a post-recall protocol test to determine students’ visual content gain and/or recall. Overall, the keyword captions group responded to visual questions about the video just as well as the no captions group, and somewhat better than the full captions group. In addition, the no captions group significantly outperformed the keyword and full captions groups on visual recall. Based on eye-tracking data, the more one looked within the captions field, the less one gained visually from the video. These results have implications for how one designs L2 video lessons with captions.
References
Bird, S. A., & Williams, J. N. (2002). The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics, 23(4), 509–533. https://doi.org/10.1017/S0142716402004022
Bisson, M.-J., Van Heuven, W. J. B., Conklin, K., & Tunney, R. (2014). Processing of native and foreign language subtitles in films: An eye tracking study. Applied Psycholinguistics, 35(2), 399–418. https://doi.org/10.1017/S0142716412000434
Guillory, H. G. (1998). The effects of keyword captions to authentic French video on learner comprehension. CALICO Journal, 15(1–3), 89–108. https://doi.org/10.1558/cj.v15i1-3.89-108
Horz, H., & Schnotz, W. (2010). Cognitive load in learning with multiple representations. In J. L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory (pp. 229–252). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.013
Jones, L. C. (2009). Supporting student differences in listening comprehension and vocabulary learning with multimedia annotations. CALICO Journal, 26(2), 267–289. https://doi.org/10.1558/cj.v26i2.267-289
Jones, L. C., & Plass, J. L. (2002). Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. Modern Language Journal, 86(4), 546–561. https://doi.org/10.1111/1540-4781.00160
Lee, H., & Mayer, R. E. (2018). Fostering learning from instructional video in a second language. Applied Cognitive Psychology, 32(5), 648–654. https://doi.org/10.1002/acp.3436
Markham, P. L., & Peter, L. (2003). The influence of English language and Spanish language captions on foreign language listening/reading comprehension. Journal of Educational Technology Systems, 31(3), 331–341. https://doi.org/10.2190/BHUH-420B-FE23-ALA0
Mayer, R. E. (2002). Cognitive theory and the design of multimedia instruction: An example of the two-way street between cognition and instruction. New Directions for Teaching and Learning, 2002(89), 55–71. https://doi.org/10.1002/tl.47
Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511811678
Mayer, R. E., & Moreno, R. (2010). Techniques that reduce extraneous cognitive load and manage intrinsic cognitive load during multimedia learning. In J. L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory (pp. 131–152). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.009
Mestres, E., & Pellicer-Sánchez, A. (2019). Young EFL learners’ processing of multimodal input: Examining learners’ eye movements. System, 80, 212–223. https://doi.org/10.1016/j.system.2018.12.002
Montero Perez, M., Peters, E., Clarebout, G., & Desmet, P. (2014). Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning & Technology, 18, 118–141.
Montero Perez, M., Peters, E., & Desmet, P. (2015). Enhancing vocabulary learning through captioned video: An eye tracking study. Modern Language Journal, 99(2), 308–328. https://doi.org/10.1111/modl.12215
Montero Perez, M., Peters, E., & Desmet, P. (2018). Vocabulary learning through viewing video: The effect of two enhancement techniques. Computer Assisted Language Learning, 31(1–2), 1–26. https://doi.org/10.1080/09588221.2017.1375960
Pellicer-Sánchez, A., Tragant, E., Conklin, K., Rodgers, M., Serrano, R., & Llanes, A. (2020). Young learners’ processing of multimodal input and its impact on reading comprehension an eye tracking study. Studies in Second Language Acquisition, 42(3), 577–598. https://doi.org/10.1017/S0272263120000091
Perego, E., Del Missier, F., Porta, M., & Mosconi, M. (2010). The cognitive effectiveness of subtitle processing. Media Psychology, 13(3), 243–272. https://doi.org/10.1080/15213269.2010.502873
Rodgers, M. P. H., & Webb, S. (2017). The effects of captions on EFL learners’ comprehension of English-language television programs. CALICO JOURNAL, 34(1), 20–38. https://doi.org/10.1558/cj.29522
Sweller, J. (1999). Instructional design in technical areas. Camberwell: ACER Press. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer. https://doi.org/10.1007/978-1-4419-8126-4
Teng, M. F. (2020). Vocabulary learning through videos: Captions advance-organizer strategy, and their combination. Computer Assisted Language Learning, 35(3), 518–550. https://doi.org/10.1080/09588221.2020.1720253
Vanderplank, R. (1988). The value of teletext sub-titles in language learning. English Language Teaching Journal, 42(4), 272–281. https://doi.org/10.1093/elt/42.4.272
Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language, Learning & Technology, 11, 67–86.
Wright, A. (2010). Pictures for language learning. Cambridge: Cambridge University Press.
Yang, J., & Chang, P. (2013). Captions and reduced forms instruction: The impact on EFL students’ listening comprehension. ReCALL, 26(1), 44–61. https://doi.org/10.1017/S0958344013000219