Intersemiotic mismatch in memes: a study of machine translation output from English into Portuguese

• Autor(a) principal: Simões, Pedro Rezende
• Data de Publicação: 2022
• Outros Autores: Pires, Thiago Blanch
• Tipo de documento: Artigo
• Idioma: por
• Título da fonte: Revista Letras Raras
• Texto Completo: https://revistas.editora.ufcg.edu.br/index.php/RLR/article/view/1148
• topic: Multimodality; Machine translation; Memes; Intersemiotic mismatches

This study presents findings on the use of Google Translator output for multimodal contexts. Development and evaluation of machine translation tend to focus on the linguistic component, while manual exploration of text-image relations in multimodal documents remains scarce. Therefore, this article aims at describing some text-image relationships in memes automatically translated from English into Portuguese. The methodology involves the selection and analysis of 100 memes found on Instagram and Facebook pages and their intersemiotic relationships both in English (as a source text) and in Portuguese (as a target text). Among the memes analyzed, 73% resulted in correct translations, 17% had errors with no intersemiotic mismatches, and only 10% showed linguistic deviations that altered text-image relationships for a meme. From these 10% of mismatches, patterns were identified, such as i) misspelled words with additive relations; and ii) unknown words with homospatiality. Finally, the results show that the automatic translation of some memes, whose semantic text-image relations share greater congruence, introduce more mismatches compared to those in which this does not happen.