Enhancing Visual Learning with Keyword Captioned Video

An Eye-Tracking Study

Authors

  • Linda C Jones University of Arkansas
  • Freddie J Jennings University of Arkansas

DOI:

https://doi.org/10.1558/cj.20284

Keywords:

captioned video, cognitive load, cognitive theory of multimedia learning, eye tracking, keyword captions, listening recall, split-attention effect, visual recall, vocabulary gain

Abstract

This current study examined the effects of full captions (n = 37), keyword captions (n = 38), and no captions (n = 37) on students’ ability to gain pictorial knowledge from a 5 minutes and 52 seconds French language video. After completing a pre-vocabulary recognition test to determine the level of equality between groups, students watched the video and immediately completed a post-visual recognition test and a post-recall protocol test to determine students’ visual content gain and/or recall. Overall, the keyword captions group responded to visual questions about the video just as well as the no captions group, and somewhat better than the full captions group. In addition, the no captions group significantly outperformed the keyword and full captions groups on visual recall. Based on eye-tracking data, the more one looked within the captions field, the less one gained visually from the video. These results have implications for how one designs L2 video lessons with captions.

Author Biographies

  • Linda C Jones, University of Arkansas

    Linda C. Jones (PhD, University of New Mexico) is Professor of Language Area Studies and Chair of the Department of World Languages, Literatures and Cultures at the University of Arkansas. Her research investigates language learning through the use of instructional technologies but also includes French Colonial Mississippi History.

  • Freddie J Jennings, University of Arkansas

    Freddie J. Jennings (PhD, University of Missouri) is a teaching assistant professor of Communication and the director of the Center for Communication Research at the University of Arkansas. His research investigates the way individuals process messages to build knowledge and form opinions.

References

Bird, S. A., & Williams, J. N. (2002). The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics, 23(4), 509–533. https://doi.org/10.1017/S0142716402004022

Bisson, M.-J., Van Heuven, W. J. B., Conklin, K., & Tunney, R. (2014). Processing of native and foreign language subtitles in films: An eye tracking study. Applied Psycholinguistics, 35(2), 399–418. https://doi.org/10.1017/S0142716412000434

Guillory, H. G. (1998). The effects of keyword captions to authentic French video on learner comprehension. CALICO Journal, 15(1–3), 89–108. https://doi.org/10.1558/cj.v15i1-3.89-108

Horz, H., & Schnotz, W. (2010). Cognitive load in learning with multiple representations. In J. L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory (pp. 229–252). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.013

Jones, L. C. (2009). Supporting student differences in listening comprehension and vocabulary learning with multimedia annotations. CALICO Journal, 26(2), 267–289. https://doi.org/10.1558/cj.v26i2.267-289

Jones, L. C., & Plass, J. L. (2002). Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations. Modern Language Journal, 86(4), 546–561. https://doi.org/10.1111/1540-4781.00160

Lee, H., & Mayer, R. E. (2018). Fostering learning from instructional video in a second language. Applied Cognitive Psychology, 32(5), 648–654. https://doi.org/10.1002/acp.3436

Markham, P. L., & Peter, L. (2003). The influence of English language and Spanish language captions on foreign language listening/reading comprehension. Journal of Educational Technology Systems, 31(3), 331–341. https://doi.org/10.2190/BHUH-420B-FE23-ALA0

Mayer, R. E. (2002). Cognitive theory and the design of multimedia instruction: An example of the two-way street between cognition and instruction. New Directions for Teaching and Learning, 2002(89), 55–71. https://doi.org/10.1002/tl.47

Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511811678

Mayer, R. E., & Moreno, R. (2010). Techniques that reduce extraneous cognitive load and manage intrinsic cognitive load during multimedia learning. In J. L. Plass, R. Moreno, & R. Brunken (Eds.), Cognitive load theory (pp. 131–152). New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511844744.009

Mestres, E., & Pellicer-Sánchez, A. (2019). Young EFL learners’ processing of multimodal input: Examining learners’ eye movements. System, 80, 212–223. https://doi.org/10.1016/j.system.2018.12.002

Montero Perez, M., Peters, E., Clarebout, G., & Desmet, P. (2014). Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning & Technology, 18, 118–141.

Montero Perez, M., Peters, E., & Desmet, P. (2015). Enhancing vocabulary learning through captioned video: An eye tracking study. Modern Language Journal, 99(2), 308–328. https://doi.org/10.1111/modl.12215

Montero Perez, M., Peters, E., & Desmet, P. (2018). Vocabulary learning through viewing video: The effect of two enhancement techniques. Computer Assisted Language Learning, 31(1–2), 1–26. https://doi.org/10.1080/09588221.2017.1375960

Pellicer-Sánchez, A., Tragant, E., Conklin, K., Rodgers, M., Serrano, R., & Llanes, A. (2020). Young learners’ processing of multimodal input and its impact on reading comprehension an eye tracking study. Studies in Second Language Acquisition, 42(3), 577–598. https://doi.org/10.1017/S0272263120000091

Perego, E., Del Missier, F., Porta, M., & Mosconi, M. (2010). The cognitive effectiveness of subtitle processing. Media Psychology, 13(3), 243–272. https://doi.org/10.1080/15213269.2010.502873

Rodgers, M. P. H., & Webb, S. (2017). The effects of captions on EFL learners’ comprehension of English-language television programs. CALICO JOURNAL, 34(1), 20–38. https://doi.org/10.1558/cj.29522

Sweller, J. (1999). Instructional design in technical areas. Camberwell: ACER Press. Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York: Springer. https://doi.org/10.1007/978-1-4419-8126-4

Teng, M. F. (2020). Vocabulary learning through videos: Captions advance-organizer strategy, and their combination. Computer Assisted Language Learning, 35(3), 518–550. https://doi.org/10.1080/09588221.2020.1720253

Vanderplank, R. (1988). The value of teletext sub-titles in language learning. English Language Teaching Journal, 42(4), 272–281. https://doi.org/10.1093/elt/42.4.272

Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language, Learning & Technology, 11, 67–86.

Wright, A. (2010). Pictures for language learning. Cambridge: Cambridge University Press.

Yang, J., & Chang, P. (2013). Captions and reduced forms instruction: The impact on EFL students’ listening comprehension. ReCALL, 26(1), 44–61. https://doi.org/10.1017/S0958344013000219

Downloads

Published

2022-10-20

Issue

Vol. 39 No. 3 (2022)

Section

Articles

How to Cite

Jones, L. C., & Jennings, F. J. (2022). Enhancing Visual Learning with Keyword Captioned Video: An Eye-Tracking Study. CALICO Journal, 39(3), 350–373. https://doi.org/10.1558/cj.20284

Most read articles by the same author(s)

1 2 3 4 5 6 7 8 9 10 > >>