“Comic Books: An Evolving Multimodal Literacy”

by Taylor Quimby | Xchanges 7.1

Consider the Speech Bubble

Aside from the textual content, the size of the text may indicate that dialogue is being spoken at a raised or hushed volume. The font of the text is sometimes used to indicate accents or vocal qualities. The color of the text may change in order to indicate emotional state of the speaker. In addition, all of these qualities can also be conveyed by the structure of the balloon itself. Todd Klein, who lettered for the visionary series The Sandman, was asked to create unique letter designs for a number of characters. On his website, he describes how the writer Neil Gaiman had specific ideas about each character’s balloon style: Delirium’s was meant to represent a mad variety going in and out of focus; Despair had a rough balloon edge to denote a ragged, rough voice; and Destruction, when he finally appeared, had an extra bold border to denote a loud, booming voice (Klein).

In an effort to capture sensual qualities that can’t be communicated in a strictly visual sense (and would be cumbersome to describe using language in the limited space available) comics use a multimodal combination of linguistics and visuals to represent other modes of meaning (particularly audio meaning) synaesthetically (McCloud 134). Perhaps the most fascinating device of comic multimodality is the panel. “The panel,” McCloud explains, “acts as a sort of general indicator that time or space is being divided” (99). Like speech bubbles, panels can come in any number of shapes and sizes, and those variations can affect how a comic is read. A long, silent panel may indicate more time has passed than a shorter panel with the same content (McCloud 101). A full-page illustration could indicate a moment of great narrative importance is occurring, influencing the reader to spend more time viewing its contents. In this way, the panel not only dictates time as it progresses in the narrative, it dictates how much time a reader ought to spend on any given sequence.

Panels spatially change or influence the narrative of a comic. When we process sets of visual, linguistic, and synaesthetic modes of meaning spatially in context with one another using panels, the narrative of comics is formed. This multimodal relationship between sets of information contained in panels is what Cope and Kalantzis call intertextuality. “Intertextuality,” they explain, “draws attention to the potentially complex ways in which meanings, such as linguistic meanings, are constituted through relationships to other texts, either real or imaginary, to other text types … to other narratives, and other modes of meanings” (30). Cope and Kalantzis argue that the world is being experienced in an increasingly multimodal fashion where meaning-making is essentially a process of integration. Because comics, more than any other form of print literature, resemble the cultural and technological move towards multimodality, they may have a unique teaching potential. Scott McCloud, who sees no limit to what the medium can achieve artistically and otherwise, was hired by Google in 2008 to create a comic book explaining their new web browser — so popular was the result that one New York Times writer called the move on Google’s part “akin to hiring Paul McCartney to write a jingle” (Gustines). But if Google is at the forefront of media innovation, then perhaps a future where comic books play a role in education isn’t so far-fetched.