Bitter Wind: A Greek Tragedy | AR/MR Case Study

Bitter Wind lens screenshot foursquare

Bitter Wind is a short AR/MR adaptation of the Greek tragedy Agamemnon.

The HoloLens version is available now. A mobile version is in progress.

Bitter Wind transforms the user’s space into a room with a mysterious mural, which contains clues to the Agamemnon mythos. As they pace the room, users occupy the POV of the grief-stricken queen Clytemnestra, who mourns her sacrificed daughter. By following directions from Bitter Wind’s ghostly hologram, users unlock the mural through HoloLens triggers:

  • Gaze targeting
  • Spatial colliders
  • Air tap
  • Object recognition of AR tag pottery shards (accompanying 3D or paper download)

My team at Fabula(b) built Bitter Wind with Unity, Vuforia, and Maya LT.

Thanks to this project, I was one of nine women, worldwide, chosen for the SH//FT Spotlight: Microsoft “Women in Mixed Reality” initiative. Click here for my Spotlight profile.

My Role

I conceived of, wrote, and designed Bitter Wind. I funded 100% of the project’s costs by winning an arts grant and an incubator residency; both were extremely competitive. I hired and managed two programmers, two technical artists, and a sound designer to build Bitter Wind under my creative direction. I authored all communications and marketing, and ran all playtests. Fabula(b), the company I launched, owns Bitter Wind in full.

Design Problems

Bitter Wind was my answer to a few design problems:

(1) People crave meaningful participation in fantastic stories, but they’re terrible authors. Since the ancient Greek festival of Dionysus, people have craved meaningful participation in a great story. It’s a core dynamic of the human experience and a through-line of my work.

But, as my PhD research found, interactive design can fall prey to a common misconception. Too often, designers assume “meaningful participation” means users can impact the storyline. For a lot of people, “inventing a great new story” is a pain point. Moreover, countless brands and story franchises will reject a solution based on users messing with a famous narrative.

So how can we create interactivity that feels meaningful, but is authentic to the original story? In designing Bitter Wind, I used the affordances of AR/MR headsets in two key ways:

  • Users trigger each story beat with embodied action. They match tangible pottery shards to holograms, pace from window to window, and gaze in a specific direction. Users already occupy a character’s visual POV—involving the full body in each trigger significantly enhances the sense of being a narrative character.
  • The headset’s limited field of view becomes an asset. Most new story beats happen several feet and 120° or more away from the previous beat. This positioning means users must walk and look all around to discover the new element, thereby creating curiosity and embodied investigation.

My design maximizes the unique aspects of AR/MR headsets to create full-body interactivity that feels necessary and meaningful. And I ask users to discover clues to the original mythos, rather than inviting them to write a new version.

(2) Can AR/MR bring something truly new to a familiar story? Whether I’m designing a digital experience or deciding which Shakespearean scene fits best in a haunted underground tunnel, I start with a question: why do I need this tool to tell this story?

To answer this question, first I identify what’s unique about the tool I’m using. AR/MR headsets have a particularly captivating affordance: users can see and hear digital things inserted into the physical world that other people in the room can’t. (For the most part—some sounds are audible).

My second step is to identify the story elements that are best—or only—brought to life with the tool I’m using. What’s been overlooked in past versions? What can I add that’s new and delightful?

Another great (and also copyright-free) source for my project would have been one of Hamlet’s scenes with his ghost father. Or Macbeth—the witches, Banquo’s ghost, or the “Is this a dagger I see before me” monologue all involve one character seeing things others might not see.

I decided against Shakespeare because I’d just shipped a short Macbeth 2D side-scroller, Something Wicked. And in addition to broadening my audience, I wanted a more complex experience than the obvious answers of “hallucination” and “ghost.” On some level, users know that what they’re looking at are holograms. This nuance—especially because users frequently forget other folks can’t see the holograms—seemed to me very much like an intense memory.

One important inspiration for Bitter Wind was a Chicago production of Agamemnon, which had a memorable scenographic element. The play takes place in front of Agamemnon and Clytemnestra’s palace door. In this production, the door opened to reveal “Iphigenia” scrawled all over the interior walls—the name of Clytemnestra’s daughter, sacrificed ten years earlier.

I was struck by the potency of this Clytemnestra’s grief, an element often glossed over in discussions of Agamemnon. She’s portrayed as powerful, murderous, angry, and even lecherous—but little is made of her sadness. I began researching, and learned that parents who’ve lost children often experience the world as if their lost child is still in it.

In other words, I used AR/MR headsets for Bitter Wind because only this technology captures this complex note of the human experience—the disorienting sensation of experiencing your own world with elements in it you know are not there,* and that only you can see.

On the mechanics level, I wanted to emphasize how AR/MR merge the digital and embodied physical realms. So I used AR tags as the primary storyline driver. The tags are tangible pottery shards, downloadable as a free paper PDF or a paid 3D file that prints pieces. (The 3D puzzle fits together in a very satisfying way). So not only do users blend their body with the story by handling the pieces, they also walk into the digital mural to match each tangible piece to its hologram.

I also wanted to create an embodied sense of how long Clytemnestra has been waiting in her house for Agamemnon to return, so she can murder him. For ten years, it’s all she’s wanted. So even though I added tension with a voiceover saying “hurry,” there is no time limit to each trigger. Users can wander the room as long as they want before moving on. And I had the programmers build spatial colliders across from each other and gaze targeting aimed at the room’s many windows. These design choices mean that to move the narrative forward, the user must pace across the room repeatedly, looking out windows.

(3) Funder priorities. To fund Bitter Wind’s hard costs and retain full IP, I needed to win an extremely competitive arts grant. This grant favors projects that (1) use the latest cutting-edge technology and (2) draw together radically different fields. When I applied for the grant, the incubator hosting my startup had just gotten a HoloLens. My grant application was successful in large part because I proposed to use this technology to draw together theatre, Classics, and computer science.

(4) Long-range revenue. For 2500 years, stagings and adaptations of Greek drama have had a guaranteed audience, populist and scholarly. Bitter Wind is one of only a few AR/MR headset narratives—once AR glasses are consumer-grade, I expect Bitter Wind to be widely downloaded. In the meantime, I am developing a mobile version for iOS and Android.

Measures of Success

A key goal of the project at large was to gain industry attention by shipping an engaging AR/MR headset narrative while the medium is still brand-new.

Bitter Wind’s success in this endeavor can be measured by my SH//FT Spotlight: Microsoft initiative (which earned Satya Nadella’s praise), as well as by my speaking engagements on the project at both industry conferences (Augmented World Expo) and scholarly conferences (Association for Theatre in Higher Education).

Bitter Wind also earned considerable attention at Northwestern, where I launched my company. A list of coverage is on the News page of this site.

In terms of the user experience of Bitter Wind, a key design goal was to pique the user’s curiosity about—and possibly memory of—Greek tragedy. I wanted to encourage people to seek out the text. I did not want users to walk away from Bitter Wind understanding Agamemnon in a way that could replace the text. I wanted them to finish the experience with enough questions and curiosity to use the clues we gave them and investigate more, using other resources.

It’s thrilling that all the playtesters who have tried Bitter Wind have expressed this curiosity—from the undergraduate who had never read a Greek tragedy to a leading scholar of Classics. Bitter Wind’s mysteries are activating new curiosities and connecting to playtesters’ personal experience with ancient drama.

Process

When I’ve secured the basics—tools, budget, timeline, story source, key team members, and a high concept—I work backwards from a delivery date. Even when I’m fortunate enough to have a good project manager, my theatre background means I am closely attuned to the production timeline. In theatre, the audience shows up at the time on their ticket—they won’t wait outside for two weeks because of crunch.

As with all the experiences I design, Bitter Wind began with all-team brainstorming sessions. To capture a range of ideas, I used the strategies I’ve honed as a classroom instructor, like a combination of spoken/written modes and one-on-one meetings for folks who are quiet in a group. Stanford’s d.school brainstorming guidelines were especially useful for this project.

I then filtered the brainstorming work to shape a coherent experience that achieved my design goals and fit within the constraints I’d identified. I wrote a draft of the Bitter Wind screenplay and sound/art asset list, and prototyped the scenic design.

I held another short series of production meetings, where we clarified and modified the assets given available time. I adjusted the narrative and scenic design as needed, and production began.

Across my experience design, once a build has passed a certain point—mechanics for digital design and off-book for live theatre—I transition from open collaboration to a more hierarchical creative direction. The project needs to get done, and art by committee is beige.

Challenges

By tech standards, my budget was minuscule. So from the iron triangle of design, I chose good and cheap, which meant a longer development timeline. And, while they were far from free, my independent contractors did work for less than market value to gain AR/MR headset experience.

During the build, key challenges were: managing a rapidly evolving Vuforia/Unity integration, digitally representing far distances in a small physical space, and getting object recognition to work.

I have a sixth sense for when a team member needs to be thrown a rope. My approach to management is to allow a short window for folks to solve an obstacle through independent research. If they run into dead ends or if their research strategies are ineffective, I intervene quickly with strategies that include:

  • Robust networking. I leverage my network (and my network’s network) to find helpful subject area experts. For Bitter Wind, experts included the Vuforia HoloLens point person, several Microsoft folks on the HoloLens team itself, and even a Tony Award-winning scenic designer, who helped us envision the illusion of distance in a small space.
  • Reevaluating design choices. If a design element holds us up, I consider alternatives. For Bitter Wind, we struggled with texture deformation on three holograms designed to look like marble statues. Before burning hours on UV unwrapping, I asked one of the tech artists to come up with alternate looks. He suggested ghost shaders, which not only fixed the issue immediately, but fit our aesthetic much better.
  • Persistence. I also know when a design choice is worth the pain. For example, object recognition caused significant programming confusion. But I considered it a vital mechanic—both for Bitter Wind’s thematics and for the team’s resume. Everyone wanted to demonstrate expertise in AR/MR design. In addition to mobilizing my network, I insisted the programmers pursue several Slack channels, often conveying questions and answers myself. Object recognition now works in Bitter Wind as the cornerstone mechanic.
Epilogue

* Designing Bitter Wind has led me to think about how AR/MR are disrupting definitions of presence, awareness, and even the permanence of death. This technology will intersect with the coming explosion in our aging population. A particularly interesting use case will be in new ways to preserve memory and reinvent presence—though hopefully with happier ends than Agamemnon.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s