Queensland University of Technology
Daniel Lynch, Lee McGowan and Donna Hancox
Iterative multimodality: An exploration of approaches to transmedia writing
Abstract
For creative writers, transmedia storytelling represents the opportunity to craft stories across a diverse narrative ecology often made up of print and digital elements. Increasingly, this requires writers to move beyond linguistic narrative renderings, and embrace the semiotic affordances and limitations of numerous media. While contemporary research highlights the importance of multimodality as a facet of transmedia storytelling, little research has focussed on the ways creative writers might approach such an undertaking. This paper examines several transmedia texts characterised by the role multimodality plays in establishing a narrative that, while dispersed across media, maintains narrative cohesion despite the different modal capacities of the media used. Through considering the factors required in continuing narrative consonance, including character and representation, the paper examines iterative multimodality as a framework for understanding how creative writers leverage narrative modes and interaction within and across media.
Keywords: transmedia storytelling, multimodality, creative writing
Introduction
Use of the term Transmedia Storytelling came into prominence through Henry Jenkins’ description of approaches to storytelling that exploited emerging media tools, techniques, and platforms, particularly those being utilised by non-experts or amateurs. First introduced in his 2003 Technology Review column, Jenkins’ concept has undergone continuous nuanced revision and refinement (see Jenkins 2006; 2010 and 2014), yet his early definition is still generally accepted. Transmedia Storytelling is the practice of creating and presenting a narrative that plays out across a range of media platforms, where each new element distinctively contributes to the larger whole (Jenkins 2006: 95-96). The practice continues to develop and grow in dynamic, important, and unexpected ways. It has proliferated across disciplines, modes and styles of presentation, and in opportunities for storytellers to connect with audiences (Hancox 2014a). Where transmedia storytelling incorporates the printed word as part of a larger arrangement of its textualities, consideration needs to be given to how different media choices can affect the overall shape of the story, and how that shape can influence the content of the narrative. This paper considers the practice of writing an expanding narrative across media in an emerging field of work and inquiry, and explores creative writers’ increasing ability to integrate concepts and theories of transmedia storytelling into their works.
The practice is arguably as old as the media it incorporates. Jason Mittell points to paintings of dramatised biblical scenes and characters such as Frankenstein and Sherlock Holmes whose narrative scope has transcended a single medium (Mittell 2014: 253). The assertion speaks to contentions concerning transmedia storytelling, where historical antecedents – old narratives being retold in new mediums – are conflated with the coordinated and preconceived distribution of a transmedia narrative (Jenkins 2010). It is not simply a case of telling a story and then pulling it across selected media, or choosing the media and then telling the story. The selection of media and the arrangement of the story’s textualities are intrinsically determined by the narrative (Jenkins 2006).
As expectations and levels of engagement of audiences across modes of storytelling grow, creative writers such as those discussed in this paper, whose use of the printed word features, but is not the only feature, in their narrative’s presentation, face determinations around coordinated and preconceived distribution. Creative writing, broad enough to encompass practices that are not purely text based, enables combinations of the written word with other visual and aural practices. The mechanics of those practices are readily understood and with it, the contemporary writer is better equipped to create stories that exploit the potential of ecological narrative arrangements. It is the evolving nature, inherent elusiveness, and complexity in describing the way these mediums relate to one another that require exploration. The aspect we explore in this paper is the relationship between form and content in the presentation of iterative multimodal narratives.
Adapted from social semiotics, multimodality stipulates that, in its most basic form, language (written or spoken) is only one of the tools available in communicating ideas or making meaning (Kress 2012). The prominence of multimodality in transmedia theory (see Dena 2009; Scolari 2009, 2013) highlights the interplay between discrete semiotic modes that occur across, and within, different media when they are positioned as parts of a singular narrative ecology. As Donna Hancox asserts, this ecology – the relationship between narrative, media and the expressions of semiotic modes they contain – allows the potential to create sophisticated and engaging stories that present media-specific representations of storyworlds and the characters who populate them (2014b; see also Lynch 2016). Grounded in narrative cohesion, and often greater than the sum of their parts, these creative writing practices require more than a linguistic rendering to function meaningfully beyond the boundaries of the printed page. They draw heavily from cognitive poetics, featuring varieties of mode within the physical text, and push readers to ‘invest in a narrative physically as well as cognitively’ (Gibbons 2010: 112). Literary novels encouraging physical interaction beyond page turning have long explored their limits. BS Johnson’s 1969 book in a box (The Unfortunates) and Kurt Vonnegut’s contemporaneous novel-length works (see Slaughterhouse-Five, 1969 and Breakfast of Champions, 1976) are among precedent-setting exemplars. More recent works, such as Mark Z Danielewski’s House of Leaves (2000) and Only Revolutions (2006); The Raw Shark Texts (Hall 2007); Important Artefacts and Personal Property from the Collection of Lenore Doolan and Harold Morris, Including Books, Street Fashion, and Jewelry (Shapton 2009); and Alejandro’s Zambra’s Multiple Choice (2016) enact multimodal storytelling. The reader is expected to arrange chapters, rotate the book, animate flipbook pages, traverse subversive footnotes, respond to quiz questions, select individual narrative pathways, navigate images and paraphernalia, and negotiate complex stylistic word arrangements in the storytelling. Multimodality can therefore be seen as a conceptual tool used by writers to represent, subvert and construct their subjective storyworlds.
Virtually any study that focusses on multimodality and creative writing draws from this history. Transmedia creative writing is influenced by these antecedents, but it need not exhibit the features of experimental novels. Working in the transmedial places emphasis on the logic of relationships between media as they extend or add to the story moving from one medium or iteration to another. While adaptation and retelling might form part of the process, we argue that the transmedia creative writer must be more concerned with the iterative nature of story and world-building and their organic expansion. Where transmedia creative writing practices are underpinned by interwoven aspects of literary and narrative theory and social semiotics, it is our contention that creative writers’ experimentation with multimodality – particularly as it embraces transmedial expression – disrupts, refigures, and reshapes definitions across fiction, and expands our understanding of the novel.
Iterative cyclical improvement is common in development in engineering design (Ummer et al 2014). Iteration is seen as repetition in computer science (Sulov 2016). For the purposes of this paper, we consider narrative iteration to be the process of exploiting tensions between linear and non-linear narrative design, through the use of character and world-building in narrative construction. This form of iteration draws on the affordances of temporality as noted by Genette (1980: 116-119) and views the relationship between audiences, authors, and the work as an amplification of the Bakhtinian conception of the novel (1981).
This paper explores the integration of multimodality and iteration in contemporary works that feature a narrative constructed through a variety of documents, each presenting various perspectives on or of the storyworld. It considers overlaps, tensions, and divergences across multimodal and transmedial narratives, and the increasing incorporation of diverse and rich media to build story and world in novel-length works. Four recent works, His Bloody Project (Burnet 2016), S. (Abrams & Dorst 2013), The Pickle Index (Horowitz, Quinn & Hubert 2015) and Flight Paths (Pullinger & Joseph 2007) are discussed for the purposes of illustration and analysis. As their narratives unfold within and across specific media, these works offer a range of representations of iterative multimodal narrativity.
Framing the approach
Transmedia storytelling has gained ground in theory, across increasingly varied disciplines, and in the practices of innovative storytellers, which has led to a blurring (and at times erasure) of the boundaries between academic determinations around narrative and emerging networked technologies. While there is arguably a great deal of valuable research emerging that attempts to broaden the theoretical scope of Transmedia Storytelling by examining facets including its geographic influences (Jenkins 2017), its historical examples (Freeman 2017) and the role of worldbuilding as a means of categorisation (Ryan 2016; Thon 2015), we present here, instead, a pragmatic approach that grounds Transmedia Storytelling theory in the practice of creative writing.
The first, and some of the definitive, examples of Transmedia Storytelling narratives tended to be aligned with ‘blockbuster’ or major mainstream creative works, sometimes referred to as motherships, such as a feature film or television series (see The Matrix Trilogy 1999-2003; Lost 2004-2010; Prometheus 2012; and Secrets & Lies 2014), while the other elements of the story were developed and distributed through online platforms and user interaction as a means of marketing or as extensions of the central narrative. More recent, contemporary transmedia stories are increasingly planned, created, and read as ecologies of storytelling (Hancox 2014b): an interlocking collection of media and modes of storytelling that build the whole picture from related aspects of the story, contributing to a textured, deeper, and more detailed storyworld presented through multiple points of view in ways that bring form and content together to create a single, arguably unique, aesthetic.
There are two pertinent threads to this positioning. The first is that the transmedia narrative ecology is more significant than the mere combination of its individual elements. The pieces work together to create a whole or wholes, and the whole depicts a specific storyworld. The second thread worth examining is the proliferation of points of view. Transmedia Storytelling narratives are expansive, and often strive to explore different characters’ understandings of narrative events, before or after they unfold (Jenkins 2010). These two threads are not unrelated, and, as Matthew Freeman (2017) argues, in narrative terms transmedia worldbuilding is a larger category of character creation. Moreover, an audience’s interpretation of any one storyworld is influenced by the ‘perceptive filters’ of characters who use voice to share their cognitive, emotional, or psychological understanding of the world around them in varying mixtures of implicit or explicit narration, description, or dialogue (Jahn 2007: 94). Within this tension, then, there is scope to evaluate the role representation plays in creative writing practice – through what modal means, and under which media specificities a writer can depict a storyworld populated by characters who are influenced by, and in turn influence, the world they exist in.
As a starting point, it is necessary to acknowledge that transmedia narratives are concerned with distinct media traditions (film, literature, games, and social media, for instance), and more importantly, operate at their intersection, particularly in the way ‘information gets dispersed’ across all of them (Jenkins 2010). Creators tend to privilege making deliberate choices around media inclusion that serve the story; but story, in transmedia narratives, is inexorably entangled with the storyworld and its characters: one cannot be developed without developing the other.
This understanding can be seen in the work of novelists such as Richard House (The Kills 2013) and Eli Horowitz and his collaborators (The Clock Without a Face 2010; The Silent History 2014; and The Pickle Index 2015), who make deliberate choices in their integration of modes of content delivery where the work attempts to elevate a range of character perspectives, world building and ‘reader participation’. Works like these demonstrate that transmedia narratives can be seen to inherently frame multimodality as an iterative process. They tell their story from multiple characters’ points of view; incorporate characteristics including voice and mode and platform of delivery in greater variety; and exploit the increasingly porous nature of the book’s ‘binding’ – arguably its most participative element, where the barrier between reader and creator/story blurs. Iterative multimodality, as presented here, is a way for writers to reconcile the disparate elements of transmedia writing – characters, worlds, form, and content – and to frame those elements as choices around representation: who gets to do the representing, who is represented, and what effect the means of representation may have on the story. It is necessary, then, to delineate what we mean by representation in the context of transmedia storytelling, and indeed, how we might begin to understand representation as an iterative process.
Representation
From a social semantic perspective, multimodality, as Kress asserts, places emphasis on the making of signs over the use of signs, and the overarching framework of generative meaning where any given sign is a fusion of form and meaning (Kress 2010: 54). Creative writers can frame those signs, in whatever modalities occur, as a function of narrative. Where they are constructed by characters in a storyworld, signs can be seen as part of the process of representation in a narrative ecology. Their representation can be described as the knowledge or understanding that a character is trying to convey about the world they live in. A character’s engagement in the storyworld becomes a representative activity allowing the reader to gain a better understanding of the world the character exists in. Tensions emerge where characters understand and express their understanding of their world in different ways from each other. This speaks to Kress’s view of multimodality, which it can be argued provides a theoretical foundation of multiple and varyingly expressed ‘knowledges’ or understandings (2010: 133). It is also pertinent, where we discuss the presentation of coherent narratives, whether they are held within the confines of a stable container such as a printed novel, or in fragments, iteratively, across a range of modes.
Multiplicity in points of view, of course, is not a concept unique to transmedia narratives, but many transmedia novels develop multiplicities across media and modes to establish, challenge, or conflate characters’ views of themselves, each other, and the world in which they exist. Works such as Marisha Pessl’s Night Film (2013) and The Silent History (Horowitz, Derby & Moffett 2014) present narratives where multiple (and often contradictory) points of view – multiple representations of knowledge – are expressed through maps, direct (written and audio) accounts, locative based smartphone applications, printed facsimiles of web content, and digital paraphernalia accessed through scanning technology. And while the language of multimodality is limited to the modal capacities of any specific media (animations, for example, cannot yet be displayed in a physical book), choices around modalities extend into choices around genre or form. The resulting representations that stem from diverse combinations of these elements, when multiple media are used, can be seen as cohesive and meaningful, providing there are identifiable points of view. That is, in a narrative sense, multimodal cohesion develops when identified characters are responsible for the representations.
We see this play out in Graeme Macrae Burnet’s 2016 historical crime novel His Bloody Project. Subtitled Documents Relating to the Case of Roderick Macrae, the novel’s conceit is veracity. The work relies on multiple unreliable perspectives presented across a range of text-based modes. And while it may initially appear to be difficult to define as an iterative multimodal work (it uses a fairly well-established modern polylogic epistolary approach), the number and kinds of documents presented, including maps, medical and court records, a glossary and an extract from a medical expert, speak to a nuanced consideration of narrative arrangement that aligns itself with more common multi-platform transmedia approaches. Even though a reader is not required to move them from between the covers of the novel, each document and image is positioned to provide context, an account of events, witness testimony, and direct and indirect participation in a set of gruesome murders and multiple movements between various sections of the text.
The reader is introduced to the story by a contemporary family relative exploring their genealogy. Witness accounts of the immediate aftermath of the killings in a very small, remote Scottish village set the scene, before the murderer’s own account captures the events leading up to his crimes, the criminal’s arrest, and his period in jail immediately prior to his court appearance. This is followed by an extract from the work of a ‘notable’ criminologist, and the trial as it was reported. The novel, then, fragments its central narrative across different points of view. Moreover, the views are expressed through specific forms of documentation (expert testimony, witness accounts, cartography, and so on) which, taken together, present a complexity in storytelling that limits its modal choices to the possibilities available to the characters at the time, and the likely forms available to characters according to their station in the broader world as presented in the story. There is a contradiction in this arrangement that speaks to the nuance of the iterative multimodal approach. Witnesses accounts are supplied, as with character testimonies, in a way that limits each individual contributor’s ability to provide context or to comment on the overall arrangement. These documents are filtered through the authorities’ process of recording and presenting evidence, which works towards presenting an argument for a conviction. On the other hand, the closer a character is to the events (such as the murderer’s long account), or the more power they have, or their inability to see their own bias in terms of contextualising the murderer’s actions (such as the criminologist’s contributions), the more acutely the character can affect the reader’s understanding of the events, and the motives behind them, which are deeply rooted in the social and economic dynamics of the local crofting community in 19th century Scotland. And yet, it is the most removed party who has the most power in affecting the arrangement of elements – as Burnet positions himself as a distant relative of the murderer – and whose job it is to sift through all the documents, and present them in a coherent manner. The contradiction is summed up during the interview of Mr M____, who states, ‘One man can no more see into the mind of another than he can see into a stone’ (taken from the entirely fictional Travels into the Borderlands of Lunacy; Thomson 1874 in Burnet 2016: 183). It is this sentiment that Burnet ultimately proves through his selection and ordering of documents and modalities, which work towards depicting, and indeed are influenced by, the world in which the story is set.
We can see in Burnet’s work a selectivity and logic to the choice of modalities, which demonstrates a purposeful restraint. It does not overstate the use of multimodality for the sake of experimentation or as a gimmick, but iteratively presents varying accounts, across visual and linguistic modalities, for an audience to encounter and interpret. From this stand point we can argue that, rather than look to multimodality as a generic form of sense-making, for creative transmedia writers, the more immediate question is how, and why, to leverage this notion in their creative practice. Iterative multimodality holds that these questions are best addressed in a transmedia story by asking how individual characters would most likely make modal choices in representing their knowledge – and not just because the writer has the access, the technology, or the necessary skills. It is most important that as characters interact within their storyworld, whether through dialogue, movement, or imagery, and so on, consistency in their engagement with and knowledge of the world is maintained, even where individual actions may be wildly contradictory. As we come into contact with more complex and distributed examples of multimodal transmedia works, it is possible to see that consideration of the world in which a story is set affords a transmedia writer opportunities in terms of structural narrative presentation. This brings us to the process of iteration in a narrative context, and examples of texts that iteratively present multimodal representations of storyworlds where characters occupy and attempt to represent their world.
Iteration and representation
Creative writers can draw on a range of modes of narrative development and creation to build their story iteratively, through a dialogue with the reader across shared points of engagement. These points can be constructed to grow the narrative beyond a single container, and can affect structure, presentation, and dynamic interaction with a story. Iterative approaches to narrative have a scholarly pedigree. Gerard Genette (1980) asserts that iteration forms part of the broader theorisation of the narratological device he calls Frequency. In his view, iteration is instrumental in separating narrative events from their narration and reducing them so that the temporal structure of the narration is either external to the narrated event (external iteration) or aligns with or meets the narrated event (internal iteration; 1980: 116-119). Genette positions iteration as the reduction of several occurrences of the same event into a single sentence. It is important, though, to note the formulation of iteration in relation to narrative, where its primary concern is with the temporal qualities of narrative presentation around the events that occur in a narrative. When this notion is extended beyond a singular medium or form – such as moving from written text into a postcard or other means of epistolic correspondence, a photographic collection, or short clip of film footage – the capacity for the presentation of events is extended according to the number and kinds of media used. Consequently, the events being presented or narrated take on a more complex, potentially ambiguous, nature. Iteration can then be understood as a processual element of story construction where narrative events, and the accounts of those events, can be responsive to the accounts and events preceding or following them. That is not to say that transmedia narratives focus solely on recounting events from different perspectives, but that as characters represent their world, as they engage with each other through whatever modal means they wish, the structural and temporal relationship between accounts can be manipulated to augment or contradict, and be used in other ways to create tension as their presentation occurs across different media.
The combination of iteration and multimodality therefore allows for detailed and pluralistic perspectives on character points of view and a broader level through collections of interconnected characters’ expressions as they play out across the narrative whole or wholes in creative transmedia writing. Iteration, in this case, shifts from simple linkage of narrative events to implications that, as characters go about the business of representing each other and the world in which they exist, they are able to engage in an iterative process, one that allows for response, disagreement, absence, acknowledgement, and contradiction across various representations. This dynamic is the catalyst at the centre of the JJ Abrams and Doug Dorst novel, S. (2013). The work’s conceit is that it is a library book, the fictitious novel Ship of Theseus, used by two college students who communicate with each other around the central text printed on the pages. Simultaneous narratives unfold through the use of traditional prose, foot notes, and ‘handwritten’ conversation and analysis, including communications on coasters, newspapers clippings, and photographs strewn between the pages of the physical novel. There is a rich ontological tapestry in the work, which is not so much transgressed as flattened. Across these texts, all conceived as part of a large narrative whole, there is a diverse array of modalities that work both at an individual level (such as a small card or a coffee-stained napkin) and together (when taken as a whole text) to tell a complex, metafictional narrative around the ‘fake’ novel Ship of Theseus and the people attempting to discover the truth about its mysterious author.
As the students attempt to solve the mystery, their correspondence develops an eclectic visual aesthetic. The shape of their handwriting, their unorthodox use of grammar, their pen-colour selection, and use of arrows highlighting parts of the text they discuss in its margins, all offer visual indications of the characters’ emotional states and the evolution of their relationship. The language and its visual aesthetic combine to develop characterisation, mood, and history. The sparse nature of their communication, limited as it is by the boundaries allowed by the margins, arguably results in more being unsaid, and mirrors shifting connections between form and content. When there is a need for characters to express more long-form thoughts, the transmediality of the text facilitates the movement from the physical book to other narrative paraphernalia, such as a letter or a postcard, where the allowances of those forms make room for more complex and extended, sentences.
Across the textual and narrative elements, the multimodality illuminates the complexity of textual arrangements in S.,and frames the college students’ quest as an iterative process of evolving analysis and responsive commentary. The storyworld is made accessible, or rather representations of the storyworld are made understandable, primarily through characters actively interpreting and questioning the preceding interpretations. Often the documents presented are arguably primarily for the reader to make sense of the world and build their own knowledge of the mystery, such as a newspaper report of the drowning of a man with a suspiciously relevant name. It is worth noting that the iterative process need not be presented in a linear fashion, nor is there a direct need to have characters explicitly comment on or discuss the arrangements of narrative media. Through arrangement of the iterative representations made by characters, the creative writer leverages the capacities of transmedia storytelling, playing into the relative strengths and weaknesses of the media used in a way that draws comparison between each preceding or future representation contained in the narrative ecology. Novels such as S. and His Bloody Project highlight the influence transmedia storytelling is having on the way writers can approach the novel, not just as a form, but as a media object – one that relies on, but is not restricted to, its text-centric modal capacity. This being the case, we can examine transmedia works that experiment more radically with different kinds of modal arrangements, and that move from between the covers into other narrative spaces.
Writing iterations
Eli Horowitz, Russell Quin and Ian Hubert’s The Pickle Index (2015) and Kate Pullinger and Chris Joseph’s Flight Paths (2007) offer alternate models of iterative storyworld representation. The Pickle Index (which is made up of two hardbound novels and a smart device application) uses iterative multimodality in a sequential order: that is, the disparate representations of the storyworld – both through imagery and written text – happen sequentially. In contrast, Flight Paths (a networked novel made up of a series of sequences that use animated text, audio/music, and dynamic images), begins with a linear progression like The Pickle Index. Using a visual approach and text to distinguish simultaneous narrative events, Flight Paths gradually draws disparate threads of the screen-based digital narrative together.
In the opening sequences of Flight Paths, the imagery, sound, and text work to reinforce the male protagonist Yacub’s world view as a stowaway in the landing gear of a plane. The next sequence depicts Harriet, the story’s female protagonist, in London going on an unnecessary trip to a supermarket. The divergence in their world views is illustrated with differences in music, colour palette, and typeface. As Yacub drops from the plane onto the supermarket carpark where he is found by Harriet, the colliding narrative threads are simultaneously portrayed across a split screen. This approach allows Flight Paths to offer a fluid narrative presentation. Its arrangement on the screen is as important as the text itself, which moves in accordance with the narrative, allowing the distinctions between the characters’ representations to continue, even as their narrative threads merge.
Split screen narrative and text placement in Flight Paths (Pullinger & Joseph 2007)
These texts, their structure, and the use of multimodality, are constructed around each character’s perspective, reinforcing voice and narrative development, and lending the arrangement immediacy and authenticity. Use of fonts, formatting, visual and audio cues, and language all contribute to the articulation of representations that offer abiding reflections of the characters’ inner and outer worlds.
Sequencing in The Pickle Index is centered on reader choice and level of engagement. The multimodal narrative enables multiple points of entry, and is accessible in a number of formats. The narrative follows Flora Baily and the rest of her circus troupe as they try to find their ringleader, Zolty Kornblatt, who has been kidnapped by the government of the dystopian future. The points of view alternate between Flora and reports published in a government-sanctioned newspaper. While the work is available in a collected paperback edition, it is the hardcover twin book edition and smartphone application that we wish to focus on here, specifically the difference between the two in leveraging character-oriented multimodal renderings of a narrative.
The app is designed as a government-sanctioned repository of pickle-based recipes. In creating an account, the reader/participant is given access to Flora’s ‘recipes’, which are actually her daily accounts of trying to locate Zolty. The reader also gathers reports describing Zolty’s incarceration through the newspaper . The app simulates narrative temporality, with accounts being ‘posted’ daily – though a willing reader can speed delivery by submitting their own ‘recipes’. The repository is deliberately user ‘unfriendly’, and is designed to visually repel as well as practically frustrate with clunky and slow interaction responses. It succeeds in demonstrating the shortcomings likely to be exhibited in a social media platform designed by an overbearing government. Building on the challenges the reader faces navigating between two overlapping texts, these design features bring the audience into the digital world of The Pickle Index, to experience it as a character would. The fictional designers of the app, set aside from the central narrative, are felt in every movement between the reports and Flora’s accounts. The use of temporality, transmedia and multimodality underlines the relationship between form and content, developing the increasingly immersive storyworld and growing our empathy for the characters who must navigate its obstacles.
The hardcover collection separates the government reports (News) and Flora’s writing (Snacks) into separate books. The reader is directed to begin reading with day one of book one, then day one of book two. Each chapter contains illustrations that capture narratorial tone, mood, and outlook. To augment the dialogic relationship between the texts, each picture can be placed alongside a corresponding image in the other book. This visual conversation between the texts reflects the stark differences between narrative viewpoints and language choices. The reader must physically move the books to make the connection and reveal incongruous imagery that parallels two points of view. While less immersive than the digital version, the reader is just as complicit in the narrative through their manipulation of the texts.
This haptic complicity is not so far removed from that of the S.; it is merely a degree of diegetic inclusion, where the digital artefact is considered as part of the object. The multimodal renderings evidenced in each are an invitation to explore, a signal to the audience that the complexity of narrative elements are part of a larger whole, and that whole is not the totality of the narrative text. Rather, the whole, or indeed wholes, offers a deeper understanding of the world of the story, as those who populate that world make their understandings of it accessible. It is this conversation between characters, and between characters and their worlds, using when necessary more than one mode and platform of communication, that allows an audience to be complicit in constructing the world for the characters to converse, interact, and act in.
Tensions emerge, here, between the reader who sees the character in the world and the character’s view of the world they inhabit. Where characters represent their understanding of their storyworld, and can be seen to draw on a diverse array of semiotic choices and discrete media to do so, it can result in complex and contradictory, often non-linear, assemblages. The notion of iterative multimodality can be used to facilitate storyworld representation in strategically designed, incrementally delivered parts. It allows characters, and of course the creative transmedia author, to challenge the affordances and limitations of specific media as part of a narrative ecology. More than this, it allows writers to consider characters as instruments through which worldbuilding – and the events, consequences, and tensions that might entail – can be presented across a mixture of distributed and self-contained structures.
Building worlds
The consequence of such a process requires us to place the choice of the media, and as a result the modal palette at a character’s disposal, in the characters’ hands. While characters cannot be disconnected from the narrative world they exist in, their representation gains narrative significance when they are charged with idiosyncratic plurality, individualised and contrasting ideologies, history, experience, and the ability to shape their storyworld. This view of the storyworld is deeply rooted in the Bakhtinian notion of the novel as a polyphonic structure: a collection of voices, speech types, and languages representing a diverse deployment of the kinds of utterances attached to different points on the social stratosphere (Bakhtin 1981: 263), where modally diverse representations of the world can be seen through the mechanics of a narrative as a cohesive assemblage. When the traditional physical form of a novel is ruptured (as happens in the transmedia novel), there is an arguable amplification of Bakhtin’s polyphony. This amplification signals a theoretical movement away from the ‘contained’ or ‘bound’ towards an ecological whole, one which may require consideration of storyworld representation as iterative in ways that allow for structural and narrative play, experimentation, and innovation in the way stories are presented to an audience. Iterative multimodality allows us to consider storyworld representation, through whatever mixture of modalities might occur, as examples of characterisation. That is, when characters are the architects of their world, the way they represent that world enriches an audience’s understanding of both the world and the character. Iterative multimodality, in this sense, can be seen to bring together form and content in a complex, engaging, and meaningful way: the narrative is enriched not just through a mixture of representations, but through a mixture of ways of representing.
In each of the texts this paper has examined, we can see that media and modal choices are steeped in the ideologies of the characters who make them (or for whom the choices are made). As we have done here, framing characters as the architects of their storyworlds, it is the audience – the readers, the viewers, the ‘manipulators’ – who do the actual worldbuilding. This relationship to the ‘text’ points to the kind of collaboration between stories and those who consume them that has historically underpinned creative writing as a practice. As author Neil Gaiman argues, the job of the writer is to provide the ‘raw code’ of a story, and when an audience considers the emotion, the sensations, the meaning they derive from a story, the audience will realise, to paraphrase Gaiman, that the reader does all the building and making (Gaiman 2016: 45).
If characters are architects and the reader a builder, where does that leave the writer? How does an author provide the ‘raw code’ of the storyworld? Semiotician and novelist, Umberto Eco, argues that ‘although the structure of a novel’s world – the setting for the events and characters of the story – is fundamental to the writer, it often must remain imprecise for the reader’ (Eco 2011: 15). Taking this notion further, Geir Farner (2014) argues that storyworlds exists beyond the story, and that as much as a story makes explicit elements of the world, doing so must also imply that there is more: a history, a geography, the knowledges and understandings that would exist beyond the narrative in question. The role of the creative transmedia writer, then, is in locating and ring-fencing an area of a storyworld, one that contains characters and imbues those characters with voices which represent their world, as their voices are shaped by their inclusion in that world. In this act – and, as we have argued, the act itself is not limited to a linguistic capacity – the writer becomes, guide, shaper, and curator, compiling, organising, and exhibiting information. Creative transmedia writing, for all the distinct and intersecting aesthetic traditions it encompasses, is tantamount to stitching multimodal assemblages for characters to represent their understanding of the world so that an audience may build it.
Conclusion
Iterative multimodal ecologies enable the transmedia writer to work with and combine diverse semiotic modes, leveraging them within and across a range of media and platforms, and to do so with the purpose of creating modal choices in narrative construction. Giving a character voice does not limit that voice to words. The increased complexity of overlapping and interacting semiotic modes extends creative writing practice. It creates greater opportunity to affect tone, mood, a sense of place, and a developing relationship with characters. The tension between the characters and the world in which they exist can be leveraged as an agent of meaning-making because these are not binary constructs. A writer cannot develop one without developing the other, and when the character/world relationship is expressed through multimodal renderings the complexity and the nuance of the relationship only increases.
Beyond this, iterative multimodality represents the playground in which creative writing stretches the theoretical edges of narrative as we currently understand it. Increasingly, as digital technologies are adapted and exploited for storytelling purposes, the definition of creative writing is likely, if not to expand, then at the very least, to be reassessed. By examining the ways in which different media interact, and more precisely the way their semiotic capacities overlap, the creative transmedia writer contributes to an as yet undefined understanding of what the future of creative writing might look like.
Works cited
Daniel Lynch completed his PhD at Queensland University of Technology, where he teaches digital and creative writing, narrative studies, and transmedia storytelling. His research into transmedia writing can be found in journals such New Writing, and his fiction can be found in Scum Mag, Tincture Journal, The London Journal of Fiction, Hypertext Magazine, and elsewhere.
Lee McGowan is a Senior Lecturer at Queensland University of Technology. He is a practitioner-researcher working with digital narratives and football (or soccer). His recent non-traditional research outputs include locative literature projects through Street Reads and Story City, and a work of experimental theatre at Queensland Theatre Company. He won the Brisbane City Council Lord Mayor’s Helen Taylor Community History Award to develop a research project focussed on the history of women’s football in Brisbane and Queensland.
Donna Hancox is a Senior Lecturer and leading researcher in Creative Writing and Literary Studies at Queensland University of Technology. Her research interests include digital writing and publishing, transmedia writing, and transmedia activism. She has been involved in a range of regional community storytelling projects; was awarded the prestigious Leverhulme Visiting Fellow at Bath Spa University (2013); led Writing the Digital Futures, a three-year project funded by Arts Queensland (2013-2016); and was awarded a Queensland-Smithsonian Fellowship in 2017.
Return to Home Page
Vol 21 No 2 October 2017
http://www.textjournal.com.au
General Editor: Nigel Krauth. Editors: Kevin Brophy, Enza Gandolfo & Julienne van Loon
text@textjournal.com.au