The Experience of Literature in the Hypersphere: A Design Research Project

Just as a new medium never completely replaces a previous one, a new media sphere never extinguishes the precursor. But it opens up new possibilities. The ebook, as the digital counterpart of the still beloved print book, never met the possibilities brought to literature by the hypersphere. Accordingly, it did not convince the market. A design research project investigated into a complementing literature experience that fully exploits the potentials of digital communication. The transdisciplinary group consisted of researchers from an art school, a business school and from different industries. The team designed the BUX app, inspired by Debray’s (2000)concept of media spheres, Bourdieu’s (1995)objectified cultural capital and the Situationists’ vision of the playful experience of culture in cities. The paper presents the digital tour guide and shows, how theoretical concepts from different age can stimulate contemporary design processes.

Keywords: city walk; cultural capital; digital storytelling; literature; media innovation;media sphere;multimedia; situationism; tour guide; transdisciplinarity

It is Martin Suter calling, the Swiss novelist. Wondering about this honour, I am picking up the phone to hear what he wants. I am surprised when he tries to warn me about a conspiracy in the Swiss banking industry. It takes some time before I realise that Martin Suteris reciting a sequence from his novel Montecristo, published in 2015, in which Max Gantmann speaks to Jonas Brand in a video message (p. 229 – 234).

Figure 1: The user receives a call from the Swiss author Martin Suter.

When Peter James 1993published his thriller Hoston two floppy disks, he was accused of killing the novel. As it turned out, the accusations were wrong. In 2017, the American book trade celebrated “print is back” and in the more cautious Switzerland, one was speaking of “a small comeback of the printed book” (von Bergen, 2017). If print is back, where has it been?

With the first “Internet-only novella” published by Stephen King in the year 2000 (Dubner, 2000), the question whether the ebook would replace the print version raised little enthusiasm. Publishers’ fears were, of course, related to the dump prices Amazon tried to establish for ebooks in countries without price fixing (Ehling, 2017). Publishers fought amazon at courts – and won (Pope, 2017). Today Amazon is doing quite a good business selling ebooks from self-publishers, mainly in the US (Ehling, 2017), but in the bigger non-English book markets such as Germany, France, Japan or China, ebooks possess a market share of not even ten per cent (Global eBook report, 2017).

The ebook may have brought a revolution for self-publishers but it did not replace the print book. A new medium never completely substitutes the precursor, but a new media sphere offers new opportunities: If people are still in love with print books, how can we use the new opportunities brought to the reading of literary texts by the technologies of the hypersphere? This is the question we would like to answer here.

In a transdisciplinary design research project, we developed a complementing experience of literature which takes into account the possibilities of digital communication: the BUX app. Our work was not just inspired by similar attempts and user studies, but also by theoretical concepts. In this paper, we will present our reflection and show how the work of Debray (2000), Merzeau (1998), Bourdieu (1995, 1986)as well as the experimentations of the Situationist International have been integrated into a contemporary design process.

In a first step, we will shortly introduce design research as a contemporary form of project-based investigation in a dynamic and complex environment. We will then discuss a bit further the failure of the ebook and present two products that extend the book in its traditional form. In a next step, we will turn to the mentioned theoretical concepts and how we drew implications from it for our project. Finally, we will introduce the BUX app as our proposal of a complementing form to experience literature in the hypersphere (Debray, 2000; Merzeau, 1998).

DESIGN RESEARCH

Design research is young. It was institutionalized in the 1960s and can be seen as an answer to increasing complexity in society: interdisciplinary issues, hard to grasp, that defy classification and processing by a single discipline define the global landscape (Churchman, 1967). In a post-industrial age, design practice moved beyond the mechanical art of manufacturing to “value-creating activities, experiences, services and digital platforms” (Tonkinwise 2016, p. 83). With the rise of new challenges design developed from being an object of research to becoming a research method(Bredies 2016). Bruce Archer defined design research in 1979as the search for and acquisition of knowledge related to design and design activity. Frayling (1993)has often been cited as differentiating between

  • design as a subject in research (research into design),
  • design as a target of research activities, i.e. the research for the improvement of design products and processes (research for design)
  • and design as a method of research inquiry (research through design).

Wolfgang adds the perspective to see design as research: “Design practice a sa gain in knowledge” (Wolfgang 2016, p. 74; italics in original). Design practice by itself is however not seen as research if it lacks reflection: the element that turns search into research (Glanville 2016).

The potentials of design research have to be evaluated in light of technological advances and their fast development, constantly delivering new options, being transferred into practice (Wolf 2016). Artefacts stemming from design research are necessary to materialize new ideas and thought processes: “Design researchers’ strength, as opposed to other scientists, is their ability to visualize” (ibid., p. 64).

The two aspects – complex, urgent societal issues and the speed of technological development – taken together, bring the need for transdisciplinary research. These practical problems reach beyond disciplinary boundaries which also constitute knowledge limits when only a segment of reality is perceived. Design research is not just transdisciplinary, but project-based and concerned with real-life problems: Affected stakeholders are integrated into the process of problem definition and the joint development of solutions (and with this, design research comes close to Action Research). Designers are trained to be sensitive to understand the world from the perspective of others and to work with experts and other stakeholders in different phases of a project and to integrate their knowledge, experience and understanding (Findeli, 2010; Held, 2016).

In this process, the research question is a design question. For our project, the design question was: How to design a complementary literature experience that meets the contemporary possibilities of digital communication?

To find answers to this research and design question, we’ve on the one hand put together a transdisciplinary research team that combined researchers from different universities and from practice. The team consisted of two business partners, the local tourism office[i]and a local interaction design agency[ii]. They collaborated with the two academic partners, a chair for media and literary studies at a Swiss business school and researchers and students from the department of audiovisual storytelling at the local art school. Authors, publishers and readers/users from the field of literature have been integrated at different stages of the research and design process.

Beyond this, the project was characterised by the ongoing integration of desk research, user studies, the assessment of technological capabilities and design activities. Further, communication and exchange across different disciplines were challenging and the establishment of a common language took time.

In this paper, we will present the conceptual sources that contributed to the development of the final artefact: a digital tour guide for the city of Zurich, based on literature that takes place within the boundaries of the city.

THE DIGITALIZATION OF LITERATURE

In a first step, we will discuss how the ebook, which was the result of media convergence (Marcinkowski, 2013), was developed as a first digital version of the book. It is our basic assumption that people (particularly from non-English countries) have not been convinced by the simple transfer of the written word into its digital version. Therefore, we will present two alternative digital offers that required a more ambitious approach. In a third step, we will then present the three theoretical approaches that inspired our project.

Ebooks: an unconvincing story

When Peter James 1993 published his thriller Hoston two floppy disks, he was accused of killing the novel. Speaking on a panel together with Apple founder Steve Jobs two years later, predicting that ebooks would catch on when they became more convenient to read, he was astonished by the amount of outrage his statement caused (Flood, 2014).

Even before James offended print-lovers, the 80s saw the launch of interactive fiction (also called text adventures), such as The Hobbit(1982) or Robert Pinsky’s Mindwheel(1984). In these experiences, the user progressed by typing in instructions (Bogost, 2010). The term “Internet-only novella[s]” or “e-novella” was later founded in 2000, when the American writer Stephen King decided to publish the two novels Riding the Bullet and The Planton the internet. His intention: to become “Big Publishing’s worst nightmare”(Dubner, 2000).

It was about that time that the first commercial e-readers were sold (Keating, 2013). It took another seven years until Jeff Bezos’ Amazon launched the Kindle to improve “something as highly evolved and well suited to its task as the book” (Jeff Bezos cited in Pope 2017). Accompanied by the global recession, people started to believe in Peter James’ prophecy that technological advances would herald the end of the book as we have known and loved it” (Pope, 2017).

However, in 2018 the printed book is anything but dead. After reaching a peak around 2013 (Preston, 2017), sales of e-readers and ebooks have slowed in several European markets resulting in almost negligible market shares between 5 and 8 per cent in bigger book markets such as Germany, France, Japan, or China (Global eBook report, 2017). Even in the US and Great Britain, where at some point ebooks accounted for almost a third of sold books, sales declined continuously for three years now (Ehling, 2017).

Not just print lovers but particularly publishers[iii]seemed to somehow be relieved about this trend[iv]and literary critics? came up with different theories to explain the decline: “Screen fatigue” (Sweney, 2017), prices (Pope, 2017), “creature comforts” (Ehling, 2017).

However, experts also state that these numbers are delusive as they would not include self-published ebooks – with a big share running through Amazon – which is not registered with ISBN numbers and hence do not become included in the statistics. In 2017, the trade fare Digital Book World in New York published data collected by the self-publishers. The results show an impressive shadow economy of more than a billion dollars from self-publishing we have not been aware of until today (Böhnel, 2017).

As complex as it may be, people turning away from ebooks can also be traced back to simple financial reasons: Amazon tried to establish low prices for ebooks to dominate the market. The big publishing houses, among other strategies with Apple involved in some of them, fought Amazon at courts – and got the right to set prices for ebooks. And they set them high. In consequence, ebook prices have climbed in recent years and now often cost the same and, in some instances, even more than physical copies (Pope, 2017).

So, after 17 years of publishing digital books, what we see is a mixed picture: People do appreciate ebooks’ accessibility, easy storage and some comparably easy digital features (as adaptable text size or change to audio mode) but many people simply still love printed books[v]. The printed book apparently has something appealing, be it “creature comforts” (Ehling, 2017)or silence (Kelly, 2018)– or both.

Beside these charismatic characteristics of the printed book, the ebook, in addition, seems not to be a convincing answer to digitalization – neither is the audiobook, which has been celebrated as the “fastest growing sector in [UK] digital publishing” most recently by the Publishers Association (2018). Book publishers made the same mistake as journalism (and other industries): Facing the digitalization, they just digitized their product without fully meeting the potentials of the digital world, i.e. without reflecting on the specificity of digital technologies, on their USPs. This results in the fact that most ebooks are more or less the same as their print counterpart – just digital (Ruppel, 2010). Arnaud Nourry, chairman and CEO of the Hachette Live Group, called the ebook self-critically a “stupid product”: It is exactly the same as print, except it’s electronic. There is no creativity, no enhancement, no real digital experience[vi](Arnaud Nourry cited in Harsimran 2018, emphasis added). The question is not whether an ebook can be as “good” as its print version, but what can I do specifically with a digital device that I would not be able to do with a printed book.

In 2010,, enhanced ebooks appeared to be a solution in the pursuit of interactivity (Ruppel, 2010). Today we know that the enhanced version neither found a convinced audience. Nourry speaks of the failure of the entire industry: “We’ve not done very well” (Arnaud Nourry cited in Harsimran 2018). He ascribes the collective failure to the lack of competences and sense for the digital. Acquiring three video game companies, he goes for quickly acquiring knowledge and multidisciplinarity. Based on these assets he strives for getting “beyond the ebook” and to “offer different experiences to [his] consumers” (Arnaud Nourry cited in Harsimran 2018).

To some extent, this is also the way we think about the digital future of the reading of literary texts, but maybe beyond digital isn’t quite accurate. Maybe truly digital would fit it. So how would such truly digital experiences look like?

So far, what we have learned is that people appreciate the traditional reading experience and stick to it. So, we are rather looking for a complementary than a substitution product.

This leads us to our research questions: How should a digital experience be designed that complements the reading experience of a book and at the same time fully meets the possibilities of digital storytelling?

For the project, we limited our focus to novels and with this to fictional or at least semi-fictional stories.

Aside from ebooks and enhanced versions, other formats have been developed that received less attention. We are going to introduce two examples identified in our research that provide alternative approaches to digitalize literature: The Swiss documentary producer Docmine’s Videobooks and the German author Julie Zeh’s enhanced print book Unterleuten(2016).

Docmine’s Videobooks: literature-based documentaries

Docmineis a creative studio based in Zurich and Munich with origins in documentary filmmaking. For their product Videobooks, the company won several international awards, including two German eBook awards (2014; 2016) and the Best of App Store award 2014. The Videobook app is available for iOS and Android and offers a collection of interactive publications. Each Videobook offers non-fictional, linear storytelling consisting of videos, text, picture galleries and interactive graphics which all merge into one storyline. Knowledge transfer is taken to the digital age, be it history, science, sport, biographies or even stories for children.

Screenshots of Docmine's work

More screenshots of Docmine's work
Impressions from two different Videobooks, offered for iOS and Android tablets by Docmine

Examples of published stories are: “Friedrich Dürrenmatt: In memory of a great author”, “Chaebols & Chabolas – The struggle for work” or “Death on the Matterhorn: A race to the top”. As a genre, the app could be called a very-enhanced-ebook with the text being only one form of storytelling (video as the dominant form). The positioning is closer to digital documentaries than to literature. This is however also related to technological reasons: The app can’t be published in the epub format and must, therefore, be offered in app stores as Google Play or Microsoft Surface. According to Patrick Müller, director and producer at Docmine, this hinders the addressing of a literature-oriented audience and the establishment of a sustainable business model.

Juli Zeh’s Unterleuten: when a book leaves its pages

Juli Zeh published her socio-critical novel in 2016, introducing the (mostly) fictional, very idyllic village Unterleuten in Brandenburg, Germany. When a big investor arrives in town, introducing the plan to construct a huge wind farm, the social balance among Unterleuten’s inhabitants falls apart, since long-time residents and newcomers act on their purest self-interest. The novel consists of six parts of eight to 13 chapters, each chapter told from the perspective of a particular inhabitant. The reader learns how everyone acts on certain assumptions about his/her fellow citizens and how all these assumptions turn out to be wrong. The novel shows how there is no truth but just perspectives.

With regard to our project, we have not so much been interested in the finesse of the story but how the story continues – in other books, on websites or even on Twitter and Facebook. Zeh not just published a web presentation of the (fictional) village and of other organizations involved in the story (e.g. a restaurant or the local society for bird protection) but she even published a life guide related to the story with the German title “Der Erfolg” and using the pseudonym “Manfred Gortz”. And she didn’t stop there: She wrote Amazon reviews for “Der Erfolg” and published several social media accounts for her protagonists. The movie adaption is being produced this summer in Germany (Stern, 2018).

With the net of digital and analogue references, Zeh expands her book to a point where reality and fiction blur, a possibility which constitutes one of the most striking specificities of digital technologies. It is an overall innovative approach of digitalization of fiction.

The two examples show how practitioners experiment with new forms that complement literature in its original form – either still complementing a print book or just referring to it on a conceptual level.

THEORETICAL APPROACHES

We will now briefly present how we derived implications from Debray’s (2000)concept of media spheres (extended by Merzeau (1998)), Bourdieu’s objectified cultural capital and the Situationists’ connection between architecture and the experience of culture in cities for our own design of the contemporary experience of literature in the hypersphere (Merzeau, 1998).

Video- and hypersphere: new medium, new possibilities and requirements

When Régis Debray formulated his “Introduction à la médiologie” in 2000, he foresaw that the book, even though in our eyes a basic form of memory, would change its shape into immaterial forms as the electronic book. In his view, the change he predicted was motivated by the possibilities offered by the video- and hypersphere. Debray describes the video- and hypersphere as heralded by the grafosphere, which brought books to the people and allowed the transfer of knowledge and myths. According to Debray the grafosphere, which emerged with the invention of printing around 1450, sped up historical rhythms as revolutions, trends or generations. The precursor of the grafosphere of course was the logosphere, which was a time of rhetoric, epic, poetry and theatre (ibid.).

The videosphere, which followed the grafosphere starts in the 1960s with (colour) TV becoming a mass media. The videosphere was dominated by audio and video technologies. Body and appearance would again outplay the word and immediacy culminate in live broadcasting. Merzeau (1998)describes the relatively short videosphere as the prelude to the more durable but unstable hypersphere, characterized by digital, convergent communication, network effects, big data, algorithms, and playful, participating users.

The possibilities brought to us by the hypersphere have started to shape literature indeed during the past years (after the ebook): from crowd-funded [vii] to collective book-writing [viii], from multilane stories [ix] to novels (supposedly) written by algorithms [x], from transparent?? customers to stories in permanent beta [xi].

Just as a new medium would never abolish the previous one, a new media sphere would never replace the precedent. In a process of restructuring and negotiating the old and the new media spheres would instead end in a state of non-arbitrary interdependence (Debray, 2000).

According to Debray (2000), the transition between media spheres is related to technological implications as well as to socio-political parameters. During a sequence of latency (which for example for the letterpress took two generations) the existing regime seems to benefit from the transition (ibid.). However, as we know, the transition between media leads eventually to a transfer of power[xii] and social change. With the change, we begin to realise the existence of the ancestral milieu because we start to recognise its boundaries and weaknesses. Grief and fear determine our thinking: this is the time when industries were hoping for the digitalization to be an ending phenomenon. Only after mourning, our mediological observations become clearer and objective again (ibid.).

As mentioned above the redistribution of power comes with fear, grief and jealousy. In the field of literature, this process is underway: publishers have lost their publication monopoly and their power to select authors from wannabes. “Real authors” are threatened by self-publishers or algorithms and lost their authority in the process of change. Book publishing today has become mostly a matter of community management.

We will see how the BUX app is not just taking advantage of the potentials of digital storytelling but further constitutes a community management and marketing tool for authors as well as for publishers. And in the next chapter, we will show how an additional opportunity offered by the hypersphere unfolds its potential not only for authors and publishers but also for city marketing: Geotagging.

Bourdieu’s cultural capital: visibility through geotagging

Associating media with time and place has always been natural for people. In the past, this association was obvious in more tangible forms such as writing the date and place where a picture had been taken on the back of the print (Luo, Joshi, Yu, & Gallagher, 2011). Today users can provide geographical identification metadata using a map interface or by adding geographically relevant keywords to published media. Technological alternatives are for example digital cameras or smartphones equipped with a GPS receiving sensor or standalone GPS receivers (Luo et al., 2011).

This process of adding geographical identification metadata to various media such as photographs or video but also websites or social media posts is called geotagging. The data that complements media usually consists of “latitude and longitude coordinates, though they can also include altitude, bearing, distance, accuracy data, and place names, and perhaps a time stamp” (Luo et al. 2011, p. 1).

The relation of information and a physical place has gained increased attention in recent years, both in research and commercial applications. Geotagging can help users find location-specific information, news, websites or other resources (Ahlers & Boll, 2008). It can tell users the location related to content on platforms such as Instagram and vice versa show media relevant to a given location on platforms as Google Maps.

The information may be of diverse nature and supplied by “common” people (non-experts). Information can also be current and conversational about live events or major happenings. (Luo et al. 2011). Twitter, Facebook, Instagram, Google or Snapchat offer their users to complement their posts with geographical identification metadata. For some platforms, this information can be extracted and leveraged in order to understand social activities. Journalists and researchers mine this kind of (big and small) data for their analysis (e.g. Bhroin, 2015; Rauchfleisch & Metag, 2016)and coverage in case of events such as earthquakes, typhoon or terrorist attacks (e.g. Rogers, 2013).

With its technological capabilities geotagging provides a digital tool not just for photographers but also for theoretical concepts: In appendix three of the prologue of his work “The Rules of Art” (1995), Pierre Bourdieu systematically analyses the novel “L’Éducation sentimentale. Histoire d’un jeune Homme” by Gustave Flaubert (1869). In his analysis Bourdieu shows how the author consciously intertwines the story and protagonists with scenes in the city of Paris. According to Bourdieu, the scenes aren’t chosen randomly: He describes “the geographical triangle” in which the story takes place (Bourdieu 1995, p. 40): the corners of this triangle “are represented by the world of business ([…] the ‚Chaussee d‘ Antin‘, the Dambreuse residence), the world of art and of successful artists ([…], the ‚Faubourg Montmartre‘, with L’Art Industriel and the successive residences of Rosanette) and the student milieu ([…] the ‚Quartier Latin‘, the initial residence of Frederic and of Martinon)” and “can be seen [as] a structure which is quite simply that of the social space of Sentimental Education” (Bourdieu 1995, p. 40; italics in original). Each of the three corners is represented by an existing area, building or neighbourhood in the city. Bourdieu goes even further when he describes how the social career of Flaubert’s protagonists moves them in different directions through the city:

Thus, in this structured and hierarchized space, ascending and descending social trajectories are clearly distinguished: from south to north-west for the former (Martinon and, for a while, Frederic), from west to east and/or from north to south for the latter (Rosanette, Arnoux). The failure of Deslauriers is marked by the fact that he never leaves the point of departure, the neighbourhood of students and failed artists (place des Trois-Maries). (Bourdieu 1995, p. 43; italics in original)

Bourdieu describes the connection of the already existing socio-cultural meaning of places in the city with meaning that’s produced when integrated into a piece of literary work as the structuring of social space. In a bidirectional process, the sociocultural meaning of geographical places is transferred to the story and vice versa: the existing geographic place is charged with cultural meaning from the fictional story.

In this process, cultural capital in the objectified state becomes linked up to geographical places: Bourdieu (1986)differentiates three forms of cultural capital: the embodied state[xiii], the objectified state and the institutionalized state[xiv]. In the objectified state, cultural capital takes the form of cultural goods such as “pictures, books, dictionaries, instruments, machines” (ibid. 17). With the structuring of social space, the objectified cultural capital (in form of a story respectively a book) becomes bound to a geographical place. In his drawing of the “The Paris of Sentimental Education” (Bourdieu 1995, p. 41), Bourdieu visualizes the cultural capital created through the story.

With geotagging, we can transfer the capturing and visualizing of objectified cultural capital into the digital. This approach however only works for stories which relate to existing places/cities/areas and disclose it – or at least provide indications (as for example Martin Suter’s Montecristo (2015)).

We will later show how we use geotagging in the BUX app in order to link literature and the city and to make the link visible. Cultural capital turns into economic capital (Bourdieu, 1986), when the city’s image becomes charged with the visibility of its cultural capital and attracts more tourists (Ashworth & Voogd, 1990).

The Situationist International: situation-based experience of culture and architecture

Turning from Bourdieu’s objectified cultural capital and geotagging to the Situationist International (SI), we have to open up for a more holistic perspective: The SI perceived the potential of architecture, design, urbanism and people’s movement within these structures to free society from capitalism, functionalism and mass production. “Situationists promised that their architecture would one day revolutionize everyday life and release the ordinary citizen into a world of experiment, anarchy, and play” (Sadler 1998, p. 69).

The group, founded in 1957, consisted mainly of members from the International Movement for an Imaginist Bauhaus and from the Lettrist International. The European (ex-)artists[xv]and intellectuals – none of the main player were designers or architects by profession – strived for societal change. Leading figures were Lettrist Guy Debord, Asger Jorn, founder of the Imaginist Bauhaus, and Constant Nieuwenhuys from pan-European COBRA[xvi](Sadler, 1998). Their ultimate goal was to reconstruct the city, “ending the capitalist contest for space and prioritization of circulation in order to organize the city for the enrichment of everyday life” (Sadler 1998, p. 117). As the “only truly human parts of life” they saw poetry and dream (Colle, 1948). In the “city of unitary urbanism” not capital and bureaucracy but participation and socio-cultural activity should become central drivers. Most of the leading situationist concepts have been already proposed between 1954 and 1956 in Potlatch, a low-budget Lettrist review of politics and art, which announced the work and the visions of the SI: psychogéographie (psychogeography),détournement(diversion), dérive (drift), situations (situations) and urbanisme unitaire (unitary urbanism) (Sadler 1998; Potlatch nr. 1,2 9, 10, 11, 14, 27).

The five concepts share the aspired, critical (physical and cognitive) examination. Psychogeography combines subjective and objective modes of studying the city. A psychogeographic drift altered the situationists‘ perception of the urban environment. At the same time, it completed the city’s taxonomy.The drift should not be confused with classical notions of the stroll. It was a conscious combination of chance and planning that reached various stages (Sadler, 1998). Détournement, or diversion, can be defined as the rerouting and hijacking of the societies pre-existing aesthetic elements (Potlatch nr. 6, 16, and 27). In constructed situations, people and their environment would develop a performance that glimpse into an “improved future based upon human encounter and play” (Ford 1995; Potlatch nr. 7, 14). A situation would be „ephemeral, without a future“ (Knabb, 1981). In the unitary urbanism, the SI “foresaw a city constituted of grand situations, between which the inhabitants would drift” endlessly (Sadler 1998, p. 117), a giant playground for the homo ludens where “architecture would merge seamlessly with all other arts” (119).

The program and vision developed by the Situationists were ambitious and uncompromising, “condemned itself to failure” (Sadler 1998, p. 106). What inspired us for our project is the Situationists’ concept of a close relation between the experience of the city and the personal examination of culture with the homo ludens, the playful citizen, at the centre of it. Of course, the Situationists would never tolerate the above-mentioned economic potential to transform cultural capital into economic capital. But while searching for inspiration for our design process, the combination of the three concepts worked well for us. In line with the Situationists’ values, the BUX app, however, is free of charge for its users.

In a next step, we will now present the BUX app in detail, which integrates the implications stemming from the three theoretical approaches and from other existing products.

THE BUX APP

Our research into existing digital products that complement literature and into transferable theoretical concepts was combined with several qualitative (potential) user studies to explore users’ needs, experiences and opinion. All three sources helped us to nourish the design process, which was split into the development of the actual app and the development of the storytelling concept on the content level. Technological capabilities and available resources[xvii]set the boundaries for the development process.

As a result, BUX is a digital tour guide to Zurich based on literary works. The free iOS app currently offers 12 tours through the city (in autumn, two additional tours will be added). Each tour is based on a story that takes place in the city. GPS-based navigation guides the walking user through different neighbourhoods. The storytelling is multimedia and modular. The author (if still alive) becomes the user’s tour guide. Otherwise, a known personality that relates to the deceased author becomes her/his replacement. On the walk, the user learns about the author, her/his relation to the city and about the specific literary work, generally a novel. All of this happens in the context of the city, the neighbourhood that surrounds the user on her/his walk [xviii].

The literary works themselves are not being touched and there is no intention to replace literature in its traditional form. The BUX app is meant and was designed as an extension and a complement to the reading culture.

The app was voted Best of Swiss Apps 2017 in the categories “functionality” and “augmented/virtual reality”. In August 2017, more than 7’200 people have downloaded the free iOS app.

Exploring the potentials of the video- and hypersphere

The cultural experience offered by the BUX app takes place at the crossroads of digital and analogue worlds. BUX communicates information on the digital level but further encourages the user to look around and (newly) discover her/his real environment on the walk. This is how the app not just addresses tourists but also local users.

In its digital storytelling, the BUX app applies all current forms of digital communication brought to us by the video- and hypersphere [xix]: text, audio, image, video, 360° video (VR), augmented reality (AR), geotagging, interactive and gamified features as well as social sharing. Putting together a tour, the composer puts the content he or she wants to communicate first and decides afterwards which “wrapping” to choose – form follows content. With this we experiment with finding new forms for given content. As we don’t limit our storytelling to one medium – a book, a movie, or a game – but feel free to choose the best-suiting form for each point resulting in a modular and flexible composition of a story. The result is a true intermedia experience [xx]that complements the very focused activity of reading a book.

Experience culture in an urban environment

By combining the silent and motionless reading experience with the actual experience of urban space and the spatialization of literature, the BUX app tries to allow for a true extension of traditional reading of literary texts: It goes a step further than previous and only descriptive research on space and literature such as the work of Franco Moretti (1999)by allowing for mobility and “lived” space. It supports the Situationist dream of transforming passive readers or spectators into active city drifters who would be able to reappropriate the urban space as their own.

The combination of information about the literary work, the author and the city should work for people who know the story and those who don’t. The user can decide if (at all) he/she wants to read the book first – and learn more about it on the tour – or afterwards – maybe inspired by the experience.

Visibility for Zurich’s cultural capital

Homescreen of the BUX app
Homescreen of the BUX app with the three areas “tours”, “explore” and the personal library.

In the second of three areas of the app[xxi], the user can explore her/his surroundings beyond the curated tours: The Explore area (in German called “Entdecken”, see Figure 3) assembles several points of interest (POI) from the 12 tours and complements them with personal recommendations from the tour guides (authors and artists). In an augmented reality application (see Figure 4), the user can scan her/his environment through the smartphone’s camera. The app indicates the closest POI, listing names, direction and distance. The user’s environment charges itself with cultural capital (Bourdieu, 1995)which is made visible and approachable. The app not only includes traditional and established references but further constructs new ones by charging geographical points with cultural meaning stemming from literary work. An example is a local Google building which is related to the well-known Zurich reformer Huldrych Zwingli.

The augmented reality application can literally be seen as a fictional lens that allows its wearer to see the city from a cultural or literary perspective – not just imagined but actually visible.

Impressions of the BUX app's Explore function
Impressions of the BUX app’s Explore function: POI are indicated (showing direction and distance), surrounding the user. The city’s cultural capital turns visible.

Community management and marketing

The city’s cultural capital turns into economic capital (Bourdieu, 1995)when its extended visibility can be transferred and charge the city’s brand and its national and international image. A culturally charged reputation holds the potential to attract (more) tourists and their spending into the local economy (Ashworth & Voogd, 1990).

Not just the local business can turn cultural into economic capital, but potentially also publishers and authors of the featured stories: for them, the BUX app can become a tool for marketing and community management. Merzeau (1998)stresses the importance of interaction, participation and networking as central characteristics of the hypersphere which fundamentally changed the publishing industry (not just literature but journalism likewise): Readers, or users, turn into the decisive collective and the controlling authority. More and more readers not just interpret but intervene (Probst & Trotier, 2013). Authors are expected to open up to their readership, let it participate in the writing process and provide insights into their life or their activities. The public lecture has been a precursor of “community management”, just as communities on Facebook, Twitter or Instagram are its contemporary form. For the authors and publishers who did collaborate with us – we also received some rejections, particularly from publishers – the opportunity to receive an additional channel to communicate with existing and potential readers of course constituted a central motivation.

Interactivity and participation

Selfie of a user wearing an AR mask from the BUX app
Selfie of a user wearing an
AR mask from the BUX app

Interactivity and participation are not done with community management – particularly if communication just happens one way. Users want to become part of the story, they wish to live, to experience it. Immersion is an important keyword of our time. Technological capabilities that allow for two-way communication and interactivity were brought to us by the hypersphere (Merzeau, 1998). The BUX app doesn’t allow users to influence the course of the story or its content but it offers opportunities to become part of it: AR-selfie-masks (see Figure 5) or social sharing [xxii] are two examples of how users can engage with the story on the walk. Additionally, we are working with pseudo-interactive features that for example stage a fake-call from the author Martin Suter (described in the introduction of this paper (see Figure 1)). Even though the call is fake, the format can still deploy an immersive effect.

Although working here and there with quizzes or small surveys, the lack of a compelling gamification approach today many non-gaming applications apply to increase engagement with the product (Knudson, 2017)is another weakness of the app. The reason is that corresponding extension of the concept would have over-complicated the design process to a degree the project available resources did not permit. The lack of resources was also the reason why we could only develop the iOS but no Android version [xxiii].

REFLEXION AND FUTURE RESEARCH

The BUX app is a digital tour guide to Zurich based on literary texts. It is the result of a transdisciplinary design research project that was inspired by the theoretical work of Debray (2000), Merzeau (1998), Bourdieu (1995, 1986)and the Situationist International. The free iOS app offers 12 tours through the city of Zurich. On these city walks, the user simultaneously explores the city and a literary work of art. GPS-based navigation guides the walking user through different neighbourhoods. The storytelling combines most contemporary forms of digital communication offered by the hypersphere (Merzeau, 1998): text-only, pictures, audio, video, 360° video (VR), augmented reality features (AR) and interactive elements. An augmented reality feature makes the city’s objectified cultural capital (Bourdieu, 1995)visible and accessible. On the walk, the user learns about the author, his/her relation to the city and about her/his work. All of this happens in the context of the city, the neighbourhood that surrounds the user on his/her walk. There is no intention to abolish literature in its traditional form but to complement it: The BUX app combines the silent and motionless reading experience with the actual experience of urban space. The reader becomes an active city drifter who reappropriates the urban space as her/his own.

With these properties, the BUX app doesn’t constitute a new, digital form of literature but it complements the traditional version and exploits the technological potentials of the hypersphere (Merzeau, 1998). Each tour is based on a story that has been thought and written for the medium book. In a next step, we are now planning to work with literature that will be thought, planned and written with the intention to complement it with a BUX tour. How is this intention going to change the nature of the emerging story?

We could even go one step further and implement a story that’s only being published on BUX – including some text fragments but text would only constitute the minority: would this still be literature? What part of a story has to be written to make it literature? What role plays the intention at the origin of the story? At what point does literature stop being literature?

With intermedia storytelling, we can ask the same question for films or games: Is it still appropriate to use these categories? Do we still need them? Or is it just digital storytelling applying different techniques (or media[xxiv])?

The future of digital storytelling holds additional possibilities that will further change how we tell stories: We already see simple – and often stupid – forms of personalized stories, for example in advertising. True personalization will go further: We could think of a BUX tour that flexibly adapts content to the user’s level of literature knowledge. The same way, journalistic content could adapt to citizens’ previous knowledge. Existing offers apply personalization at the article level (preferred topics or assigned keywords). Advanced offers will apply personalization at the content level with varying text for different users. Of course, such alignments have to be based on advanced algorithms that know our previous consumption in very much detail and it implies advanced automated text generation.

And then, there are first stories and other forms of artefacts (e.g. pictures or poems) that have been generated by machines and algorithms. One example are the movies that have been produced based on the scripts written by Benjamin, the world’s first artificially-intelligent screenwriter[xxv].

Whatever these and next advances will mean for storytelling and storytellers, people in the future will still love books. Print books.

REFERENCES

Ahlers, D., & Boll, S. (2008). `Oh Web Image, Where Art Thou?’. In S. Satoh, F. Nack, & M. Etoh (Eds.), Advances in Multimedia Modeling(pp. 101–112). Berlin, Heidelberg: Springer Berlin Heidelberg.

Archer, B. (1979). Design as a discipline. Design Studies1, 1(1), 17–20. https://doi.org/10.1016/0142-694X(79)90023-1

Ashworth, G. J., & Voogd, H. (1990). Selling the city: marketing approaches in public sector urban planning.London: Behlhaven Press.

Bhroin, N. N. (2015). Social Media-Innovation: The Case of Indigenous Tweets. The Journal of Media Innovations, 11(2), 89–106. Retrieved from http://www.journals.uio.no/index.php/TJMI

Bogost, I. (2010). Persuasive Games. Cambridge, MA: MIT Press.

Böhnel, M. (2017). Kleine Verlage träumen weiterhin von «digitaler Ermüdung». SRF. Retrieved from https://www.srf.ch/kultur/literatur/kleine-verlage-traeumen-weiterhin-von-digitaler-ermuedung

Bourdieu, P. (1986). The Forms of Capital. In Handbook of theory and research for the sociology of education(pp. 241–258). Westport, CT: Greenwood. https://doi.org/10.1017/CBO9781107415324.004

Bourdieu, P. (1995). The rules of art. Genesis and structure of the literary field. Stanford California: Stanford University Press.

Bredies, K. (2016). Introduction. In G. Joost, K. Bredies, M. Christensen, F. Conradi, & A. Unteidig (Eds.), Design as Research. Positions, Arguments, Perspectives(pp. 12–16). Basel: Birkhäuser Verlag GmbH.

Churchman, C. W. (1967). Wicked Problems. Management Science, 14(4), B141–B142.

Cocozza, P. (2017, April 27). How eBooks lost their shine: “Kindles now look clunky and unhip.” The Guardian. Retrieved from https://www.theguardian.com/books/2017/apr/27/how-ebooks-lost-their-shine-kindles-look-clunky-unhip-

Colle, M. (1948). Vers une architecture symbolique. Cobra,1(1), 21–23.

Debray, R. (2000). Introduction à la médiologie. Paris: Presses Universitaires de France.

Debray, R. (2003). Einführung in die Mediologie. Bern: Haupt.

Dubner, S. J. (2000, August 13). What Is Stephen King Trying to Prove?The New York Times Magazines. Retrieved from https://archive.nytimes.com/www.nytimes.com/library/magazine/home/20000813mag-king.html

Earls, N. (2017, May 11). Has the print book trumped digital? Beware of glib conclusions. Australian Broadcasting Corporation (ABC). Retrieved from http://www.abc.net.au/news/2017-05-09/has-the-print-book-trumped-digital-beware-glib-conclusions/8510984

Ehling, H. (2017, November 12). E-Books – eine Revolution, die nicht stattfand. SRF. Retrieved from https://www.srf.ch/kultur/literatur/wochenende-literatur/10-jahre-schweizer-buchpreis-e-books-eine-revolution-die-nicht-stattfand

Fenske, T. (2012). Getting Dumped. San Francisco: Coliloquy.

Findeli, A. (2010). Searching for Design Research Questions. In R. Chow, W. Jonas, & G. Joost (Eds.), Questions, hypotheses & conjectures. Discussions on projects by early stage and senior design researchers(pp. 286–303). iUniverse. Retrieved from https://www.researchgate.net/profile/Alain_Findeli/publication/235700600_Searching_for_DesignResearch_Questions_Some_Conceptual_Clarifications/links/5591abeb08ae15962d8e2ec0/Searching-for-DesignResearch-Questions-Some-Conceptual-Clarifications.pdf

Flaubert, G. (1869). L’Éducation sentimentale. Histoire d’un jeune Homme. Paris: Michel Lévy Frères.

Flood, A. (2014). Where did the story of ebooks begin? The Guardian. Retrieved from https://www.theguardian.com/books/2014/mar/12/ebooks-begin-medium-reading-peter-james

Ford, S. (1995). The realization and suppression of the situationist international. An annotated bibliography 1972-1992. Edinburgh: AK Press.

Frayling, C. (1993). Research in Art and Design. Royal College of Art Research Papers, 1(1), 1–5.

Glanville, R. (2016). Design Prepositions. In G. Joost, K. Bredies, M. Christensen, F. Conradi, & A. Unteidig (Eds.), Design as Research. Positions, Arguments, Perspectives(pp. 153–165). Basel: Birkhäuser Verlag GmbH.

Global eBook report. (2017). Global eBook Report 2017. Press release and preview selected findings.

Harsimran, G. (2018). ‘The ebook is a stupid product: no creativity, no enhancement,’ says Hachette Group CEO. Scroll.In. Retrieved from https://scroll.in/article/868871/the-ebook-is-a-stupid-product-no-creativity-no-enhancement-says-the-hachette-group-ceo

Held, M. (2016). Transdisciplinary research through design – shifting paradigms as an opportunity. In G. Joost, K. Bredies, M. Christensen, F. Conradi, & A. Unteidig (Eds.), Design as Research. Positions, Arguments, Perspectives. Basel: Birkhäuser Verlag GmbH.

James, P. (1993). Host. Victor Gollancz.

Keating, S. (2013). The rise and rise of the ebook. The Irish Times. Retrieved from https://www.irishtimes.com/culture/books/the-rise-and-rise-of-the-ebook-1.1320014

Kelly, E. (2018). Ebooks are not “stupid” – they’re a revolution. The Guardian. Retrieved from https://www.theguardian.com/books/2018/feb/21/ebooks-are-not-stupid-theyre-a-revolution-erin-kelly

Knabb, K. (1981). Situationist International Anthology. Berkeley: Bureau of Public Secrets.

Knudson, P. (2017, July). Gamification in 2017: Top 5 Key Principles. Medium. UX Planet. Retrieved from https://uxplanet.org/gamification-in-2017-top-5-key-principles-cef948254dad

Luo, J., Joshi, D., Yu, J., & Gallagher, A. (2011). Geotagging in multimedia and computer vision – a survey. Multimedia Tools and Applications, 51(1), 187–211. https://doi.org/10.1007/s11042-010-0623-y

Marcinkowski, F. (2013). Medienkonvergenz. In G. Bentele, H.-B. Brosius, & O. Jarren (Eds.), Studienbücher zur Kommunikations- und Medienwissenschaft. Lexikon Kommunikations- und Medienwissenschaft(2nd ed., pp. 210–211). Wiesbaden: Springer VS.

Merzeau, L. (1998). Ceci ne tuera pas cela. Cahiers de Médologie, 2(6), 27–39.

Moretti, F. (1999). Atlas of the European Novel: 1800-1900. New York: Verso.

Nieuwenhuys, C. (1985). Demain la poésie logera la vie. In Documents relatifs à la fondation de l’Internationale Situationniste(pp. 1948–1957). Paris: Allia.

Pope, C. (2017). Rise and fall of the Kindle: How real books are fighting back. The Irish Times. Retrieved from https://www.irishtimes.com/news/consumer/rise-and-fall-of-the-kindle-how-real-books-are-fighting-back-1.3086282

Preston, A. (2017). How real books have trumped ebooks. The Observer. Retrieved from https://www.theguardian.com/books/2017/may/14/how-real-books-trumped-ebooks-publishing-revival

Probst, M., & Trotier, K. (2013). Leser, mach’s dir selbst! Die Zeit Online. Retrieved from https://www.zeit.de/2013/06/Internet-Buecher-schreiben

Rauchfleisch, A., & Metag, J. (2016). The special case of Switzerland: Swiss politicians on Twitter. New Media and Society, 18(10), 2413–2431. https://doi.org/10.1177/1461444815586982

Rogers, S. (2013, July 10). The Boston Bombing: How journalists used Twitter to tell the story. Twitter Blog. Retrieved from https://blog.twitter.com/official/en_us/a/2013/the-boston-bombing-how-journalists-used-twitter-to-tell-the-story.html

Ruppel, P. (2010, December 27). 5 e-book trends that will change the future of publishing. Mashable. Retrieved from https://mashable.com/2010/12/27/e-book-publishing-trends/?europe=true#6TTcnXO50OqO

Sadler, S. (1998). The Situationist City. Cambridge, MA: MIT Press.

Senkel, M. (2018). Dunkle Zahlen. Berlin: Matthes & Seitz.

Stern. (2018). Drehstart: Juli Zahs “Unterleuten” wird verfilmt. Stern. Retrieved from https://www.stern.de/kultur/tv/zdf-dreiteiler-drehstart–juli-zehs–unterleuten–wird-verfilmt-8155378.html

Suter, M. (2015). Montecristo. Zürich: Diogenes Verlag AG.

Sweney, M. (2017, April 27). “Screen fatigue” sees UK ebook sales plunge 17% as readers return to print. The Guardian. Retrieved from https://www.theguardian.com/books/2017/apr/27/screen-fatigue-sees-uk-ebook-sales-plunge-17-as-readers-return-to-print

The Publishers Association. (2018). Audiobooks remain fastest growing sector in digital publishing. Retrieved from https://www.publishers.org.uk/news/press-releases/2018/audiobooks-remain-fastest-growing-sector-in-digital-publishing/

Tonkinwise, C. (2016). Everyday homeopathy in practice-changing design research. In G. Joost, K. Bredies, M. Christensen, F. Conradi, & A. Unteidig (Eds.), Design as Research. Positions, Arguments, Perspectives(pp. 83–90). Basel: Birkhäuser Verlag GmbH.

von Bergen, S. (2017, October 4). Das gedruckte Buch lässt sich vom E-Book nicht so leicht verdrängen. Berner Zeitung. Retrieved from https://www.bernerzeitung.ch/schweiz/standard/das-gedruckte-buch-laesst-sich-vomebook-nicht-so-leicht-verdraengen/story/19073183

Von Gehlen, D. (2013). Eine neue Version ist Verfügbar. Berlin & Ulm: Metrolit Verlag GmbH & Co. KG.

Wolf, B. (2016). Communicating in design research. In G. Joos, K. Bredies, M. Christensen, F. Conradi, & A. Unteidig (Eds.), Design as Research. Positions, Arguments, Perspectives(pp. 64–69). Basel: Birkhäuser Verlag GmbH.

Wolfgang, J. (2016). Text vs. artefact in design research? A strange question! In G. Joost, K. Bredies, M. Christensen, F. Conradi, & A. Unteidig (Eds.), Design as Research. Positions, Arguments, Perspectives(pp. 70–76). Basel: Birkhäuser Verlag GmbH.

Zeh, J. (2016).Unterleuten. München: Luchterhand.

[i]Zurich Tourism

[ii]dreipol

[iii]Which seems somehow counterintuitive considering that ebooks have a much bigger margin.

[iv]“How eBooks lost their shine” (Cocozza, 2017), “How real books have trumped ebooks” (Preston 2017; emphasis added)“as grey and forgettable as ebooks” (Preston, 2017). An explanation for the respire could be that Amazon accounted for a big share of ebook sales as long as it was able to set the prices independently (Earls, 2017).

[v]It’s actually interesting to see how the book persists in its original form compared for example to music, which “was hopping like a Whirling Dervish from cylinders to vinyl to cassettes, to CDs, to mini-discs to ones and zeros stored on your computer’s hard drive and then on to something even more ephemeral in the form of streamed content” (Pope, 2017).

[vi]A comparison is the emergence of TV in the sixties: In its beginning, TV was mostly filmed radio and films on TV were broadcasted cinema. It took some time until TV found its peculiarity, its USP and this is also what we saw in recent years with the digitalisation of different media products as music or journalism.

[vii]Dirk von Gehlen received the funding for his book “Eine neue Version ist verfügbar” (2013)through a crowd funding campaign. The crowd also got involved in the following writing process.

[viii]At the end of her novel “Getting Dumped”, Tawna Fenske (2012)asked her community for whom of three aspirants the heroine should opt. Based on its decision, the author wrote the sequel.

[ix]Tawna Fenske (2012)integrated some forks into her ebook “Getting Dumped”, curious for the decisions her readers would make. The different paths later in the book come back together.

[x]“Dunkle Zahlen” by GLM-3/Matthias Senkel (2018)

[xi]Publishers together with their authors are mining the data stemming from ebook readers. This data provides exact information about reader behaviour: After a specific passage, too many readers are putting the book away? Then rewrite it! Accordingly, book titles, covers or whole passages are under permanent review (Probst & Trotier, 2013).

[xii]Which we can observe, when Amazon is threatening legacy publishers with its self-publishing platforms and ebook prices. And publishers fighting Amazon at courts.

[xiii]Culture, cultivation, education: It’s acquirement “presupposes a process of embodiment, incorporation, which, insofar as it implies a labor of inculcation and assimilation, costs time, time which must be invested personally by the investor” (Bourdieu 1986, p. 18).

[xiv]Academic qualification (Bourdieu, 1986).

[xv]Constant announced himself as an „ex-artist“ at the First World Conference of Free Artists by way of a declaration of his commitment to the construction of situations (Nieuwenhuys, 1985).

[xvi]Acronym of Copenhagen, Brussels, and Amsterdam. Common front of groups from those cities against the regulation of politics and art (Sadler, 1998).

[xvii]The project was mostly funded by Innosuisse, the Swiss federal innovation agency. Additionally, both business partners, the local tourist office and the digital agency, invested own resources into the realization.

[xviii]A video that collects the multimedia-content of a particular tour with English subtitles can be seen on the BUX app’s YouTube channel or directly here.

[xix]And of course, the grafosphere.

[xx]If we consider the option of sharing the tour with friends, even a transmedia experience.

[xxi]The first and main area, in German called «Touren» (tours), contains the 12 curated city walks. The third area, which contains the user’s personal library, contains the downloaded tours and pictures take on tours.

[xxii]At the end of each tour,the user can share her/his experience with friends on different channels.

[xxiii]According to the involved programmers from the interaction design agency, the support for iOS development goes further than for Android.

[xxiv]Debray (2003, p. 47)points to the problem that the term “medium” can refer to (1) a general process of symbolisation (e.g. the spoken word, drawings or a picture), (2) a social code for communication (e.g. language), (3) a physical carrier for writing or drawing (e.g. stone, paper, CD-ROM) and (4) a dispositive for circulation (e.g. letter print or digitalization).

[xxv]In this TED talk, Ross and Oscar describe their work: https://www.youtube.com/watch?v=uPXPQK83Z_Y