An Overview of Research-creation in and with Interactive Media

Frédéric Dubois, (Internationale Filmschule Köln)

This essay contributes to and updates scholarship on practice-based research. More concretely, it situates research-creation vis-à-vis practice-based research and discusses the distinction between artistic and mediatic research-creation. The essay insists on affordances of interactive media as they are foundational for drawing accurate research results. The paper contributes to theorizing mediatic research-creation by adding the component of interactivity to it. By introducing interactive documentary (i-doc) as a creative media practice, and by providing a succinct case study of i-doc Field Trip (2019), the text engages with the notion of mediatic research-creation head-on. Disclaimer: Some passages in this paper are drawn from the author’s doctoral thesis (Dubois, 2021)

Approaches to research

In 1981, Elliott Eisner discussed ten differences between the artistic and the scientific approaches to research (1981). His paper is a historical marker, as it is one of just a few early attempts at differentiating two modes of knowledge generation. While acknowledging that art and science can be defined in a variety of ways, he goes on to untangle the two modi with dimensions such as modes of representation, criteria for appraisal, and the degree of license (authorship) employed in each. Eisner's short comparative essay can further be read as a plaidoyer for legitimising artistic approaches to research. "It is to the artistic to which we must turn, not as a rejection of the scientific, but because with both we can achieve binocular vision. Looking through one eye never did provide much depth of field" (1981, p. 8).

Interestingly, his essay is of particular currency today, where the debate about what's scientific, fact-based and true and, what's fake (as in fake news or alternative truths) is in full swing. Eisner is in no way advocating to reject truth-seeking, instead insisting on the different pages that scientific and artistic approaches to research are on:

Artistic approaches to research are less concerned with the discovery of truth than with the creation of meaning. Truth implies singularity and monopoly. Meaning implies relativism and diversity. Truth is more closely wedded to consistency and logic, meaning to diverse interpretation and coherence (1981, p. 8).

Forty years on, Eisner's text continues to be foundational for differentiating modes of inquiry. But since Eisner, there has been a plethora of literature discussing the type of knowledge that artistic approaches to research bring about (e.g., "unique, particular, local knowledge" say Balkema & Slager, 2004, p. 13; 'unfinished thinking' dixit

Borgdorff, 2010). These perspectives have not fundamentally put into question Eisner, but much more followed suit in specifying art-based research.

The 'scientific' and the 'artistic' have both evolved, branched out, and hybridized. In the social sciences today, 'scholars' have largely replaced 'scientists' loosening up the very idea of research, and by what norms research should be done. In parallel, the 'artists' have been joined by the 'designers' (UX designers, architects, audience designers, etc) in the larger context of the creative communities (Meroni, 2007) and creative industries (Hesmondhalgh, 2008) paradigms. Yet to this day, the long shade of the 'scientific' continues to have an effect on our shared understanding of what research is.

In the next section, I am defining the types of practice-based research, so as to delineate what research best applies for inquiries in interactive documentaries. The aim is that by exploring typologies of practice-based research, I contextualise my own practice, and at the same time underline the importance of integrating interactive media affordances in the research design of media inquiries.

Types of research

Practice-based research

Researchers in Australia, the UK and many other Organisation for Economic Co-operation and Development ( OECD) countries have been adopting the notion of practice-based research as one way to provide the needed depth of field. But what is practice-based research? Gauntlett (2021, n.p.) states that "Practice-based research is work where, in order to explore their research question, the researcher needs to make things as part of the process. The research is exploratory and is embedded in a creative practice". Under the umbrella of practice-based research there are competing concepts embracing the binocular vision that Eisner is referring to. For each of these denominations, there are different ontologies and epistemologies. Brad Haseman (2006, p. 98) for instance, speaks of performative research, which he defines as "an alternative to the qualitative and quantitative paradigms by insisting on different approaches to designing, conducting and reporting research". Others have used the termspractice-led research, or research through practice for describing their strategies-aimed at running truth-seeking and meaning-making alongside each other. These closely-related terms have family resemblances, particularly when it comes to methodologies.

Research-creation

For the field of media studies I prefer using the term research-creation , without denying that it is a 'close cousin' of the more general practice-based research denomination. Generally defined, "research-creation projects typically integrate a creative process, experimental aesthetic component, or an artistic work as an integral part of a study" (Chapman & Sawchuk, 2008, p. 1). Research-creation has a relatively long tradition in both French (recherche-création) and English, which not so coincidentally are also Canada's main languages. Second to none, this North-American country's research institutions have integrated and formalized this designation, among other with the support of dedicated (research-creation) funding.

Beyond the 'Canadian tradition' though, it is fair to ask why this notion should have a role to play moving forward. Some have argued that "a key step towards the specificity of 'research-creation' is the move to disseminate its knowledge in the language of practice" (Cohen, 2015, n.p.). It is true that some research-creation endeavors have helped rejuvenate and break up scholarly conventions by allowing forms of knowledge expression common to the arts (e.g. music compositions, artistic projections, photography). But there is more.

For one, when contrasted with other maker practices, such as purely technical or mechanical ones (e.g. engineering), the notion of research-creation speaks specifically to a creative work (as in: das Werk , in German) and a creative process (as in: die Gestaltung, in German). In the field of media studies, more so in creative media research, this emphasis on what is creative (understood here as art and design) is helpful, as it sets up the right context of inquiry. Secondly, research-creation leaves no doubt as to what comes first, research or creation. The term starts with the word "research", thereby foregrounding that any endeavor of this type is driven by a research question-or what is behind it: an open and investigative curiosity. In other words, a project might well be creation-led, i.e. the knowledge generated through the making of a designed work might be dominant (over that of academic literature and theory), or the research question might have been initiated in practice. But to fall within the boundaries of research-creation, the project needs to be initiated and defined within the limits of a research mindset. Paquin and Béland (2015, n.p.), drawing on Chapman and Sawchuk (2008) warn us though, that "at either end of the spectrum are those who support research for creative purposes and those who argue that creation is a form of research". This is an ongoing discussion which can only be settled by the researchers themselves, as it relates to the purpose with which they do research. In my own view, creation in and of itself is in no way research. For an endeavour to be called research, it needs to pursue a research question (which attempts to better practice, or which triggers answers which are more philosophical in nature).

Thirdly, instead of just positioning research and creation side-by-side, research-creation stresses the interplay between two approaches. The hyphen in research-creation reinforces the iterative ambition of researching with conventional research strategies and creative ones at the same time. The researcher-creator is 'walking and chewing gum at the same time'.

While the exact position of the researcher on the spectrum of research-creation might vary, even at times challenging its boundaries, this does not dispense the researcher-creator from using basic inquiry methodologies, i.e. rigorous documentation (e.g. with the help of a research diary), systematic analysis of the creative work, etc. Particularly the care put into documenting one's own creative practice will determine the researcher's ability to draw valid evidence from the practical side of the investigation. There are many methods with which creative practice can be documented, including thick description (Geertz, 1973), to name just one. What's common to most documentation though, is the aim to hold on to key descriptions and observations of creative practice processes through time. Documentation, if adequately performed, provides an access to 'the backstage' of a project. It is the memory of at times mundane actions and larger decisions by creative teams, which all serve to situate a creative practice in time and space.

Mediatic research-creation

More recently, Paquin has started distinguishing, together with Noury (Paquin & Noury, 2020), artistic research-creation from mediatic research-creation-where the latter refers to endeavors in which creative projects are bound by constraints specific to the media ecology. As I interpret these constraints from my own media practice, they relate to things like the political economy of the media, time constraints in media-making, emphasis on the product over the process, performance requirements, audience taking on a defined/target role, and short-term impact accountability.

This distinction is useful to better contextualize the specific affordances of creative media practice and to acknowledge that as a result, the findings of this type of practice might differ in sometimes decisive ways from those of artistic research-creation. In the latter, one could be researching on how to render a dance performance more compelling with the choice of colour of costumes or sound aesthetics. They would be testing and experimenting different moods. In the case of an interactive motion comics, to be distributed online as well as on-site in a museum, the team might be researching along aesthetic lines, just like in the dance performance example, with the difference that the audience design will be treated in a much more granular manner, segmenting audience on different social media platforms online and the museum-goers on site. By doing this, the makers adopt mediatic affordances, such as algorithmic selection on social media to help craft the motion comic. While in the dance performance the findings might indicate certain aesthetic preferences based on systematic testing of colour schemes in costumes, the findings in the motion comic will include data about the different target audiences and how this is integrated in aesthetic choices. In other words, mediatic research-creation is oftentimes rationally more complex than artistic research-creation, and also closer to production studies, in which technological affordances and the political economy of media play a more salient role.

By zooming-in from practice-based research to research-creation, and sharpening the lens even more to identify mediatic research-creation, it brings us in reach of research done with and/or through interactive media more specifically - the dominant form of media consumed today (e.g. social media, video games, interactive and immersive audiovisuals). It is necessary to grasp and recognize the parameters of interactive media so that as a researcher and practitioner, we get a chance to develop research projects that are state of the art in terms of methodology (i.e. the correct use of netnography for analyzing trends in viewing habits might require an appropriate reading and handling of mainstream video games).

Interactive media

Interactive media have their own set of affordances. Independently of the genre, i.e. from video games to virtual reality, from touch screens in museums to interactive fictional series, and from long form narratives on social media to digital performance art, they put the relationship to the audience in the focus. Back in 2006, Cover wrote: "The interactive and digital nature of computer-mediated communication results in several new tensions in the author-text-audience relationship, predominantly through blurring the line between author and audience, and eroding older technological, policy and conventional models for the 'control' of the text, its narrative sequencing and its distribution." (Cover, 2006, p. 140). If interactive media authorship is often distributed, creative practice happens in collaborative teams, or even in co-creation arrangements with specific audiences. Although the dialogic use of media is not new (see Brecht, with his theory of media), the social media age has multiplied the collaborative potential of media making and consuming. This might entail a qualitatively increased sense of agency for users of such platforms, and at the same time raise important questions of copyright, which are not explored in this paper.

This said, in certain forms of interactive media, "text or its content is affected, resequenced, altered, customized or re-narrated in the interactive process of audiencehood" (Ibid). This is also what in more recent literature on creative media production has been referred to as co-creation (Rose, 2018) of narratives, where the audience has a part to play in the crafting of the story itself.

As we will see in the next section, interactive documentary is such a form in which co-creation is often present.

Affordances of interactive documentary

In the case of interactive documentary (i-doc)-a non-fiction form of interactive media that is mainly browser-based-I have regrouped the main characteristics under Table 1, as drawn from my own doctoral research (Dubois, 2021).

Common characteristics of interactive documentaries

Table 1Common characteristics of interactive documentaries. Source: Dubois, 2021.

Table 1 summarizes the main affordances of i-docs in terms of product and process. Similar to other interactive media, i-docs have hypertextuality at their core.

In i-docs, makers generally create, assemble and present documentary material in a form that is native to web technologies. Interactive documentary productions are media works that typically include one or more point(s)-of-view, an interactive interface, a delinearised narrative, and at times, participatory features meant to involve citizens in the storytelling. Over the decade spanning 2007 to 2017, i-doc makers have produced and distributed their works with the help of higher education institutions, public service media, the public interest press, film and media funds, as well as film festivals.

I-docs qualify, as I argue elsewhere (Dubois, 2020), as media innovations , i.e. they are "multidimentional and risky products" (Dogruel, 2014, p. 52). I-docs are not standalone art works, although they might possess similar characteristics in terms of their experimental or exploratory nature. Unlike art works, i-docs are proper media productions, i.e. they are closer to complex crafts productions in terms of riskiness and they come alive through the relational dimension of interaction and manipulation by users (Gaudenzi, 2013). This uncertain encounter with its audience, its users, or produsers (Bruns, 2007), is what make them high-risk productions.

Field Trip

Image 1 Screenshot of i-doc Field. Source: Field Trip GbR.

Field Trip is an interactive documentary produced independently by a small team of media makers in Berlin. It was started in 2017 and released in its first version in 2019. It is accessible in open access on fieldtrip.berlin. The documentary takes the user onto the Tempelhof Field in Berlin, in what was once an airfield and is now the largest park in that metropolis. The user visits the field through the stories of 14 protagonists told in an interactive fashion. Every story is separated in scenes from which the user can jump into scenes from other stories, thereby allowing for an interactive viewing experience. Beyond viewing, the user can co-create the larger narrative of Field Trip by simply leaving an anecdote, via the hypertextual interface, on an answering machine. The team behind Field Tripcurates the best anecdotes and includes them into the Storyboxx, which is one of the protagonists of the field.

In my own doctoral research, I have pursued a research-creation approach. My research-question reflects this approach particularly well: how to account for the societal impact of interactive documentary, as much from a theoretical reflection as from a maker's understanding?

I have spent three years producing the independent interactive documentary Field Trip as an interactive producer. In parallel, I have developed my research agenda, leading up to this research question. Now, when performing the research, I had to read into the artistic work both in terms of product and process, extracting theoretical knowledge from impact, reception and documentary studies on the one hand and experiential knowledge from my tasks as producer on the other. Through the practice of interactive documentary making particularly, I was able to gain privileged and first-hand information on impact strategies and tactics and how to evaluate impact in a context of constant media innovation.

One of the most striking findings from that research, beyond the definitional work or the identification of affordances of interactive documentary, is that interactive documentary and interactive media in general, need to account for societal impact in terms of their openness. Drawing mainly from the creation (Field Trip), I was able to establish that openness in interactive media can be understood as a triad: open content, open format (interface) and open technology (Dubois, 2021). In other words, the specific makeup of Field Trip has permitted me to generate grounded theory that I could then feed back into the larger research endeavor on the societal impact of interactive documentary. Now, when compared to a free artistic creation, Field Trip was a proper production, with production deadlines, funding and constraints. Therefore, what resulted from my research is a mediatic research-creation which cannot be compared one-to-one to an artistic research-creation project of doctoral colleagues. One of the main differences is that colleagues would in general look much more at aesthetics and internal artistic processes, much more than production and utilitarian questions, such as those related to the notion of impact. I am on purpose not developing the findings on impact here, as they are exposed in long form in my doctoral thesis, available in open access (Dubois, 2021).

Conclusion

Interactive documentary is a subgenre of interactive media with which to engage in mediatic research-creation and to inform and expand this type of research-creation. It is rich in intrinsic artistic and narrative components and affordances, as well as production and process-oriented learnings. Field Trip, as only one illustration of this subgenre permits me to affirm that it is important to specify what type of practice-based research a researcher is doing. By positioning my own research as a mediatic research-creation, I was able to situate my research as part of a larger body of creative media research. What this entails for other projects moving forward, is that it is important to examine typologies around practice-based research, including those most closely related to media studies.

This paper has permitted me to learn to differentiate between more artistic and more media-related research-creation endeavours and thus to adopt a more accurate lense (which makes room for affordances of today's dominant form of media - interactive media) when approaching fieldwork.

The challenges I faced in this essay are similar to those of most qualitative enquiries, where single case studies and self-produced projects might only tell one part of a much more complex system, in which interactive media affordances are many, and their nature and aims vary much. Researching with and about a blockbuster video game or with and about an independent podcast are two different things. The motivations behind these media forms are different and the technology, authorship and co-creation affordances are diverse. In this sense, this paper is to be understood as an essay with a limited scope of research, only focusing on certain features of interactive media.

References

• Aston, Judith, Gaudenzi, Sandra, and Rose, Mandy (Eds.). 2017. i-Docs: the evolving practices of interactive documentary . Columbia University Press.
• Balkema, Annette W., and Slager, Henk (eds.). 2004. Artistic research . Vol. 18. Rodopi.
• Borgdorff, Henk. 2010. "The production of knowledge in artistic research." In The Routledge companion to research in the arts , pp. 44-63. Routledge.
• Bruns, Axel. 2007. "Produsage: Towards a Broader Framework for User-Led Content Creation". In Proceedings Creativity & Cognition (PDF). Washington, DC. 6.
• Chapman, Owen, and Sawchuk, Kim. 2008. "Research-creation: Intervention, analysis and 'family resemblances'". Canadian Journal of Communication, 37, 5-26. https://cjc-online.ca/index.php/journal/article/view/2489/2766
• Cohen, Hart. 2015. Research Creation: a scholarship of creativity. Journal of the New Media Caucus, 11(3), 48-61. http://median.newmediacaucus.org/research-creation-explorations/research-creation-a-scholarship-of-creativity/
• Cover Rob. 2006. Audience inter/active: Interactive media, narrative control and reconceiving audience history. New Media & Society . 2006;8(1):139-158. doi: 10.1177/1461444806059922
• Dogruel, Leyla. 2014. "What is so Special about Media Innovations? A Characterization of the Field". The Journal of Media Innovations, 1 (1), 52-69.
• Dubois, Frédéric. 2021. Interactive Documentary Production and Societal Impact: The Case of Field Trip . Doctoral dissertation, Filmuniversität Babelsberg Konrad Wolf.
• Dubois, Frédéric. 2020. "Media innovation and social impact: the case of living documentaries". The Journal of Media Innovations, 6 (1), 23-37.
• Eisner, Elliot W. 1981. On the differences between scientific and artistic approaches to qualitative research. Educational researcher , 10(4), 5-9.
• Gaudenzi, Sandra. 2013. "The Living Documentary: from representing reality to co-creating reality in digital interactive documentary". Doctoral thesis, Goldsmiths, University of London [Thesis]
• Gauntlett, David. 2021. https://davidgauntlett.com/research-practice/what-is-practice-based-research/
• Geertz, Clifford. 1973. "Chapter 1/Thick Description: Toward an interpretive theory of culture." The Interpretation of Cultures: Selected Essays, New York, Basic Books .
• Haseman, B. 2006. A Manifesto for Performative Research. Media International Australia , 118(1), 98-106. https://doi.org/10.1177/1329878X0611800113
• Hesmondhalgh, Desmond. 2008. Cultural and Creative Industries. In: Bennett, T and Frow, J, (eds.) The SAGE handbook of cultural analysis. Sage Publications Ltd , ? - ? (732). ISBN 9780761942290
• Gely, Raphaël. 2000. "La question de l'événement dans la phénoménologie de Merleau-Ponty," Laval théologique et philosophique 56, no 2 (2000): 361.
• Meroni, Anna. 2007. Creative Communities. People inventing sustainable ways of living . Edizioni Polidesign.
• Nash, Kate. 2021. Interactive Documentary: Theory and Debate . Routledge.
• Paquin, Louis-Claude, and Béland, Marjolaine. 2015. Dialogue autour de la recherche-création. Media/n, 11(3).
• Paquin,Louis-Claude, and Noury, Cynthia.2020. "A Small Narrative of the Emergence of Mediatic Research-Creation at UQAM and Some Proposals to Guide its Practice". Communiquer , La communication à l'UQAM, 103-136. http://journals.openedition.org/communiquer/5042
• Rose, Mandy. 2018. 4. Not media about, but media with: co-creation for activism. In i-DOCS (pp. 49-65). Columbia University Press.
• Smith, Hazel (Ed.). (2009). Practice-led research, research-led practice in the creative arts . Edinburgh University Press.