Although artistic augmented reality (AR) and mixed reality (MR) experiences have been discussed in multiple scholarly fields, when it comes to interaction design, comprehensive accounts of actual design practices are rare. To expand our knowledge of design practices, this PhD dissertation brings together choreography and interaction design, approaching MR as site-specific performance. More specifically, choreography as an expanded practice is used to encourage discussions about the creation of MR experiences in public spaces. The research project involved practice-based research and the creation of three public site-specific MR walks in Göteborg and Mölndal, realised in collaboration with an artistic team and stakeholders from the art, cultural heritage and information technology sectors. The dissertation shows that a choreographic approach provides concrete ways for crafting relations between time, space, bodies and imagination in AR/MR experiences, demonstrating that technology is imbued with site-specific power relations, ethics and aesthetics. These contributions are both practice-oriented and theoretical. They include an expansion of our knowledge of embodied interaction and its relevance to public space, an articulation of choreography as a broad practice that can be applied to technological design and a distinct set of composition practices for site-specific AR/MR experiences. Together, these contributions result in strategies and critical reflection on how AR/MR experiences may be used to reactivate public spaces and their political significance in society through critical, embodied and artistic techniques.

This paper traces different conceptions of the body in HCI and identifies a narrative from user to body, body to bodies, and bodies to more-than-human bodies. Firstly, this paper aims to present a broader, updated, survey of work around the body in HCI. The overview shows how bodies are conceptualized as performative, sensing, datafied, intersectional and more-than-human. This paper then diverges from similar surveys of research addressing the body in HCI in that it is more disruptive and offers a critique of these approaches and pointers for where HCI might go next. We end our paper with recommendations drawn from across the different approaches to the body in HCI. In particular, that researchers working with the body have much to gain from the 4th wave HCI approach when designing with and for the body, where our relationships with technologies are understood as entangled and the body is always more-than-human.  

Enter Mölndal – Kvarnbyn is a web-based mixed reality walk and museum experience that the audience experience on location in Kvarnbyn - an old industrial town in Mölndal, Sweden - with their own smartphone and a pair of headphones. It is a documentary form of storytelling with digital panoramas and micro-stories that together forms a larger story about the practical and socio-political situation in Kvarnbyn and its 19th and 20th century industrial history. The walk starts next to Forsebron, the main bridge over the rapids with a spectacular view of the waterfall. It consists of four stops to which the user is guided. At each place the smartphone screen displays a panorama image of the location. Each panorama has 2-5 windows and the user is invited to tap on the window to experience a short story. These micro stories are portrayed as short films with binaural sound. They are based on the museum’s archival material and are told by a street lamp, a washing lady, a linseed, factory workers and an eel, to name a few. In total there are 16 micros stories to be discovered during the walk. The web-based experience uses augmented (AR) and mixed reality (MR) technologies together with location-aware applications. Artistically these technologies are used to provide multiple layers of the site and unfold the potential transactions between body, imagination, senses and environment. The audience sees and feels one thing in reality, hears/sees another layer of the same place via the smartphone, and creates a third in their own imagination and thought. All information, i.e. sound, narration, videos, and animations, are created and developed on site from the archival material. They are therefore site specific and interact in a nuanced way with the physical world, the buildings around, and the situation. Duration: Ca 40 min, depending on choices made along the walk. Langauges: Available in Swedish and English. Capacity: 1 person per smartphone. Technical format: The experience runs on the smartphone’s web browser and is started by entering https://entermolndal.se. The content is streamed to the smartphone and wifi is needed during the walk. Site specific but not locked, i.e. the digital part of the experience can be started anywhere. Premiere: 16 June 2018, at Kvarnbyn in Mölndal, Sweden. Part of Mölndal Stadsmuseum’s permanent exhibitions.

Enter Mölndal – Tulebosjön is a a site-specific mixed reality walk and AR- app that lets the user explore the unique bronze age findings next to Tulebosjön (Lake Tulebo) in Mölndal, Sweden. A large system of cooking pits has been found here, with hundreds of pits in a straight line running along the top of the ridge, but nothing of it remains. With a smartphone in the hand and headphones on the head, the user is guided via an AR-app into virtual spaces that tells about the actual location, the findings and their time. The target group is children from 9 years and adults. The walk starts next to the beach at Tulebosjön and the user is guided along a path to the highest point at the slope. Along this path 8 virtual spaces can be found, only visible and able to find via the mobile. In these virtual spaces the visitor meets concrete and poetic AR models in 3D that interacts with the physical location, short info about the place and the bronze age, as well as entrances to reflect on what this system of cooking pits may have been used for, since no-one actually knows. It is an immersive experience in which the user is the protagonist, yet a walk done with open ears, open eyes and open senses. Thereby it creates an embodied experience of the place and its history. Nothing of the bronze age findings are left, but the magnificence of the place is still there. The app uses augmented (AR) and mixed reality (MR) technologies together with location-aware applications. Artistically these technologies are used to provide multiple layers of the site and unfold the potential transactions between body, imagination, senses and environment. The audience sees and feels one thing in reality, hears/sees another layer of the same place via the smartphone, and creates a third in their own imagination and thought. All information, i.e. narration, 3D models, videos, animations and sound scores are created and developed on site and based on archaeologists’ and researchers’ material. They are therefore site specific and interact in a nuanced way with the physical world, the landscape, and the situation. Duration: 20-30 min, depending on how long time spent in each virtual space. Langauges: Available in Swedish and English. Capacity: 1 person per smartphone. Technical format: An app that runs on iPhone and Android. Available free of charge at App Store and Google Play. Needs wifi for downloading the app, but no wifi is needed during the walk. Site specific, uses the GPS-position of the site in order to start. The app uses new AR-technology (ARKit and ARCore), which requires a smartphone with support for this. Premiere: 20 September 2018, at Tulebosjön in Mölndal, Sweden. Part of Mölndal Stadsmuseum’s permanent exhibitions.

Next to You at Korsvägen is a site-specific mixed reality walk, made as an interactive and private exploration, charged by an app for smartphones. The individual audience participants hold smartphones in their hands and have headphones on. It is a documentary form of storytelling with branching narratives where the user is taken to different locations at the triangular platform in search for “the new world”. The work remixes the city's visions of Göteborg as a destination with quotes from Vilhelm Moberg's novel series Utvandrarna (The Emigrants), and facts about Korsvägen. Via the smartphone’s screen and the headphones, the user encounters a critical montage of different stories about Korsvägen and related events, told through voice, binaural sound, film, text and 3D-animations, at specific locations at the platform. It is an immersive experience in which the user is the protagonist, yet a walk done with open ears, open eyes and open senses. The app uses augmented (AR) and mixed reality (MR) technologies together with location-aware applications. Artistically these technologies are used to provide multiple layers of the site and unfold the potential transactions between body, imagination, senses and environment . The audience sees and feels one thing in reality, hears/sees another via the smartphone, and creates a third in their own imagination and thought. All information, i.e. sound, narration, videos, and animations, are created and developed on site. They are therefore site specific and interact in a nuanced way with the physical world, the architecture and the situation. Duration: 35- 40 min depending on choices made along the walk. Capacity: 1 person per smartphone. People start with 3-5 minutes intervals. The app: Runs on both iPhones and Androids. Available free of charge at App Store and Google Play. Setting 1: Equipment - smartphone and headphones - is handed out, and the work is experienced during the decided opening hours. Setting 2: People download the app on their own smartphone and use with their own headphones, at any time. Premiere: September 2017 at GIBCA Extended - Gothenburg International Biennial for Contemporary Art.

Colonial practices often come unnoticed with participatory projects that involve people, places and economics. There are complex power structures involved, and nuances of ethics at play. But how to address this complexity in a constructive manner? By explicitly drawing on a convergence between the challenges addressed by participatory art and participatory design, I propose a postcolonial perspective as a valuable critical practice for understanding the dynamic power structures in participatory projects. In order to decolonise a participatory project I suggest a set of concrete questions that can be asked regularly. To be the killjoy that points to the less flattering aspects of a project is never an easy position to take. But it may be necessary if the field of art and design is to raise its awareness and contribute to an ethical change.

Colonial practices often come unnoticed with participatory projects that involve people, places and economics. There are complex power structures involved, and nuances of ethics at play. But how to address this complexity in a constructive manner? By explicitly drawing on a convergence between the challenges addressed by participatory art and participatory design, I propose a postcolonial perspective as a valuable critical practice for understanding the dynamic power structures in participatory projects. In order to decolonise a participatory project I suggest a set of concrete questions that can be asked regularly. To be the killjoy that points to the less flattering aspects of a project is never an easy position to take. But it may be necessary if the field of art and design is to raise its awareness and contribute to an ethical change.