Influence of semiotic mode on inferential multimodal reading comprehension of university students
Abstract
From the multimodal theoretical perspective of systemic functional linguistics and cognitive psychology developed by Mason et al. (2013), Parodi (2011), Parodi and Julio (2015; 2017), Parodi et al. (2020) and Bateman and Hiippala (2021); and empirical evidence, this study contributes to the limited body of research in the multimodal research line, specifically focusing on multimodal inferential reading competence. To this end, a multisemiotic text -National Institute of Statistics and Informatics (Inei) bulletin was subjected to testing, which was segmented into three independent semiotic artifacts: full original text (TC), text with only graphics (SG), and text with only verbal content (SV). The research objective was to determine which of the three tested artifacts is more efficient for the identification and justification of textual inferences. An intersubject experimental design was employed, with the participation of 235 students from the industrial engineering and social communication programs at a public university in Peru. The results indicate that the original multimodal artifact (TC) and the verbal-only artifact (SV) achieved a higher and better identification of appropriate inferences by industrial engineering students. In other words, they achieve greater success than those in the social communication program. Therefore, it provides empirical evidence of differences in inferential comprehension among students in the mentioned programs, based on the semiotic composition of the texts.
Downloads
Metrics
References
Referencias
Abraham, G. S., Rajasekharan, L., & Anirudhan, S. (2021). Investigation of inferential skills in school going preadolescents. International Journal of Health Sciences and Research, 11(7), 272-280. DOI: https://doi.org/10.52403/ijhsr.20210737
Bateman, J. A., McDonald, D., Hiippala, T., Couto-Vale, D., & Costetchi, E. (2019). Systemic functional linguistics and computation: new directions, new challenges. In G. Thompson, W. L. Bowcher, L. Fontaine, & D. Schöntal (Eds.), The Cambridge Handbook of systemic functional linguistics (p. 561-586). Cambridge, GB: Cambridge University Press.
Bateman, J. (2008). Multimodality and genre: a foundation for the systematic analysis of multimodal documents. Basingstoke; New York: Springer.
Bateman, J. A. (2014). The constitutive role of semiotic modes for the theory and practice of multimodal analysis. In C. DeCoursey (Ed.), Language arts in Asia 2: English and Chinese through literature, drama and popular cultura (p. 8-33). Cambridge, GB: Cambridge Scholars.
Bateman, J. A. (2021). Artefactos multisemióticos entre modos y medios. Revista Signos, 54(107), 842-866. DOI: https://dx.doi.org/10.4067/S0718-09342021000300842
Bateman, J. A., & Hiippala, T. (2021). Multimodality and hypertext: theoretical and empirical considerations. In 32nd ACM Conference on Hypertext and Social Media (HT '21) (p. 3-4). Association for Computing Machinery, New York, NY.
Bateman, J. A., Wildfeuer, J., & Hiippala, T. (2017). Multimodality: foundations, research and analysis – A problem-oriented introduction. Boston, MA: De Gruyter Mouton.
Boudon, E., & Parodi, G. (2014). Artefactos multisemióticos y discurso académico de la Economía: construcción de conocimientos en el género Manual. Revista Signos, 47(85), 164-195. DOI: https://dx.doi.org/10.4067/S0718-09342014000200002
Cahyaningati, D. T., & Lestari, L. A. (2018). The use of multimodal text in enhancing engineering students' reading skill. International Journal of Language Education, 2(2), 65-73. DOI: https://doi.org/10.26858/ijole.v2i2.6360
Cisneros-Estupiñán, M., Olave-Arias, G., & Rojas-García, I. (2012). Cómo mejorar la capacidad inferencial en estudiantes universitarios. Educación y Educadores, 15(1), 45-61.
Danielsson, K., & Selander, S. (2021). Multimodal texts in disciplinary education. A comprehensive framework. Cham, SW: Springer.
De Mena, C. V. (2015). Comprensión de textos y la habilidad inferencial. Exlibris, (4), 396-404.
Domínguez-Lara, S. A. (2018). Autoeficacia para situaciones académicas en estudiantes universitarios peruanos: un enfoque de ecuaciones estructurales. Revista de Psicología, 4, 43-54.
Gladic-Miralles, J., & Cautín-Epifani, V. (2018). Niveles de comprensión y su relación con la predominancia de sistemas semióticos: una aproximación a la comprensión multimodal desde el discurso académico. Estudios Pedagógicos, 44(1), 293-313. DOI: https://dx.doi.org/10.4067/S0718-07052018000100293
Gutiérrez-Calvo, M. (1999). Inferencias en la comprensión del lenguaje. In M. Vega & F. Cuetos (Coord.), Psicolingüística del español (p. 231-270). Madrid, ES: Editorial Trotta.
Instituto Nacional de Estadística e Informativa [Inei]. (2020, 10 de mai.). Indicadores económicos. Recuperado de https://m.inei.gob.pe/media/MenuRecursivo/boletines/boletin-quincenal-10_2.pdf
Jewitt, C. (2012). Technology, literacy, learning: a multimodal approach. London, GB: Routledge.
Jewitt, C., Bezemer, J., & O'Halloran, K. (2016). Introducing multimodality. London, GB: Routledge.
Julio, C., Parodi, G., & Loureda, Ó. (2019). Congruencia entre sistemas semióticos: estudio de palabras y gráficos con el uso de eye tracker. Estudios Filológicos, (63), 237-260. DOI: https://dx.doi.org/10.4067/S0071-17132019000100237
Kendeou, P., Bohn-Gettler, C., White, M. J., & Van Den Broek, P. (2008). Children's inference generation across different media. Journal of Research in Reading, 31(3), 259-338. DOI: https://doi.org/10.1111/j.1467-9817.2008.00370.x
Kintsch, W. (1998). Comprehension: a paradigm for cognition. New York, NY: Cambridge University Press.
Kress, G. (2010). Multimodalidad: una aproximación semiótica social a la comunicación contemporánea. London, GB: Routledge.
Kress, G., & van Leeuwen, T. (2001). Multimodal discourse: the modes and media of contemporary communication. London, GB: Arnold.
Lemke, J. L. (1998). Multiplying meaning: visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading science: critical and functional perspectives on discourses of science (p. 87-113). London, GB: Routledge.
Lemke, J. L. (2005). Multimedia genres and traversals. Folia Linguistica, 39(1-2), 45-56. DOI: https://doi.org/10.1515/flin.2005.39.1-2.45
Cascón, J. A. L. (2003). Conocimiento y discurso. Claves para inferir y comprender. Madrid, ES: Pirámide.
Liu, Y., & O’Halloran, K. L. (2009). Intersemiotic texture: analyzing cohesive devices between language and images. Social Semiotics, 19(4), 367-380. DOI: https://doi.org/10.1080/10350330903361059
Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns. Computers & Education, 60(1), 95-109. DOI: https://doi.org/10.1016/j.compedu.2012.07.011
Matthiessen, C. M. I. M. (2014). The multimodal page: a systemic functional exploration. In T. D. Royce & W. Bowcher (Eds.), New directions in the analysis of multimodal discourse (p. 1-62). New York, NY: Routledge.
Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge, GB: Cambridge University Press.
Meneses, A., Escobar, J.-P., & Véliz, S. (2018). The effects of multimodal texts on science reading comprehension in Chilean fifth-graders: text scaffolding and comprehension skills. International Journal of Science Education, 40(18), 2226-2244. DOI: https://doi.org/10.1080/09500693.2018.1527472
Molina, M., Vásquez-Rocca, L., & Parodi, G. (2018). Relación palabra-gráfico en un género profesional de la economía: el Informe Mensual de Estadísticas Monetarias y Financieras. Círculo de Lingüística Aplicada a la Comunicación, 76, 153-175. DOI: http://dx.doi.org/10.5209/CLAC.62503
Nurviyani, V., Suherdi, D., & Lukmana, I. (2020). Developing students’ reading skill through making multimodal inferences. English Review: Journal of English Education, 8(2), 175-182. DOI: https://doi.org/10.25134/erjee.v8i2.2998
Parodi, G. (2005). Comprensión de textos escritos. Buenos Aires, AR: Eudeba.
Parodi, G. (2010). Multisemiosis y lingüística de corpus: Artefactos (multi)semióticos en los textos de seis disciplinas en el corpus PUCV-2010. Revista de Lingüística Teórica y Aplicada, 48(2), 33-70. DOI: https://dx.doi.org/10.4067/S0718-48832010000200003
Parodi, G. (2011). La Teoría de la Comunicabilidad: Notas para una concepción integral de la comprensión de textos escritos. Revista Signos, 44(76), 145-167. DOI: https://dx.doi.org/10.4067/S0718-09342011000200004
Parodi, G., & Julio, C. (2015). Más allá de las palabras: ¿Puede comprenderse el género discursivo Informe de Política Monetaria desde un único sistema semiótico predominante? Alpha (Osorno), (41), 133-158. DOI: https://dx.doi.org/10.4067/S0718-22012015000200011
Parodi, G., & Julio, C. (2017). No solo existen palabras en los textos escritos: algunas teorías y modelos de comprensión de textos multimodales o multisemióticos. Investigaciones Sobre Lectura, 8, 27-48.
Parodi, G., Moreno-de-León, T., & Julio, C. (2020). Comprehension of Written Texts: Reconceptualizing 21st century challenges. Íkala, Revista de Lenguaje y Cultura, 25(3), 775-795. DOI: https://doi.org/10.17533/udea.ikala.v25n03a10
Parodi, G., Julio, C., & Recio, I. (2018). Cuando las palabras y los gráficos mueven los ojos: el procesamiento de relaciones causales multimodales. Revista de investigación del movimiento ocular, 11(1). DOI: https://doi.org/10.16910/jemr.11.1.5
Silva, W. P., & Marchon, A. H. (2021). Argumentação multimodal: uma proposta teórico-metodológica. Acta Scientiarum. Language and Culture, 43(1), e56894. DOI: https://doi.org/10.4025/actascilangcult.v43i1.56894
Pérez, A., Schmidt, E., Kourtzi, Z., & Tsimpli, I. (2020). Multimodal semantic revision during inferential processing: The role of inhibitory control in text and picture comprehension. Neuropsychologia, 138, 107313. DOI: https://doi.org/10.1016/j.neuropsychologia.2019.107313
Royce, T. (1998). Synergy on the page: exploring intersemiotic complementarity in page-based multimodal text. JASFL Occasional Papers, 1(1), 25-49.
Sadoski, M. (1992). Imagination, cognition, and persona. Rhetoric Review, 10(2), 266-278. DOI: https://doi.org/10.1080/07350199209388971
Sadoski, M., & Paivio, A. (2004). A dual coding theoretical model of reading. In R. B. Ruddell & N. J. Unrau (Eds.), Theoretical models and processes of reading (5th ed., p. 1329-1362). Newark, NJ: International Reading Association.
Sadoski, M., Paivio, A., & Goetz, E. T. (1991). A critique of schema theory in reading and a dual coding alternative. Reading Research Quarterly, 26(4), 463-484. DOI: https://doi.org/10.2307/747898
Schnotz, W. (2002). Towards an integrated view of learning from text and visual displays. Educational Psychology Review, 14(1), 101-120.
Schnotz, W., & Horz, H. (2010). New media, learning from. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education (3rd ed., p. 140-149). New York, NY: Elsevier.
Shah, P., & Freedman, E. G. (2011). Bar and line graph comprehension: an interaction of top-down and bottom-up processes. Topics in Cognitive Science, 3(3), 560-578. DOI: https://doi.org/10.1111/j.1756-8765.2009.01066.x
van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York, NY: Academic Press.
Vásquez-Rocca, L. (2018). Desafíos para el desarrollo de una competencia multimodal disciplinar. El caso del género informe de política monetaria (IPOM). Signa: Revista de la Asociación Española de Semiótica, 27, 1095-1123. DOI: https://doi.org/10.5944/signa.vol27.2018.21857
Vásquez-Rocca, L., & Parodi, G. (2015). Relaciones retóricas y multimodalidad en el género Informe de Política Monetaria del discurso académico de la Economía. Calidoscópio, 13(3), 388-405.
Millalén, F. V. (2014). Impacto de la multimodalidad en la comprensión lectora de textos narrativos en inglés como lengua extranjera (L2) en estudiantes universitarios. Contextos Educativos. Revista de Educación, (18), 25-41. DOI: https://doi.org/10.18172/con.2608
Villalonga-Penna, M. M. (2020). Leer en la secundaria y leer en el primer año de psicología: perspectivas de los docentes con más y con menos experiencia. Revista Colombiana de Educación, (80), 53-74. DOI: https://doi.org/10.17227/rce.num80-9591
Yum, Y. N., Cohn, N., & Lau, W. K.-W. (2021). Effects of picture-word integration on reading visual narratives in L1 and L2. Learning and Instruction, 71, 101397. DOI: https://doi.org/10.1016/j.learninstruc.2020.101397
Zhao, F., Schnotz, W., Wagner, I., & Gaschler, R. (2020). Texts and pictures serve different functions in conjoint mental model construction and adaptation. Memory & Cognition, 48, 69-82. DOI: https://doi.org/10.3758/s13421-019-00962-0
DECLARATION OF ORIGINALITY AND COPYRIGHTS
I Declare that current article is original and has not been submitted for publication, in part or in whole, to any other national or international journal.
The copyrights belong exclusively to the authors. Published content is licensed under Creative Commons Attribution 4.0 (CC BY 4.0) guidelines, which allows sharing (copy and distribution of the material in any medium or format) and adaptation (remix, transform, and build upon the material) for any purpose, even commercially, under the terms of attribution.
Read this link for further information on how to use CC BY 4.0 properly.
Funding data
-
Universidad Nacional de San Agustin de Arequipa
Grant numbers Contratado de financiamientoIBA-CS-01-2019-UNSA
6.png){kind=logo}
