In the process of working on Eva’s story in the AR environment, we experienced our own stories, these are a snapshot of these journeys:
The idea of the project Vessels of Memory is to create little virtual displays that pop up from different objects that used to belong to our main character, Eva Garrard. These virtual displays had to be insights of his inner world in a certain moment of Eva’s life; the idea is to tell one part of the story of Eva, in the same way visual arts tell a story, but using the interactive approach that this new medium offers.
After choosing the object from which we wanted to extract a story for Eva Garrard I chose a pearls necklace. As well, I chose to tell and invent a story that makes sense to me about her life. We didn´t know the whole story behind Eva and we, as a team, were interested in conserving the freedom to create and add something to the story of Eva, in this way, re-creating the character. In my case I chose to talk about the final part of her life, when she was already a widow, living a situation of abuse from his tenant. For this, I decided to give Eva a way out that emotionally could bring her back to life, her love for music, particularly for playing piano.
After deciding the story the next part of the process was the visualization of the story according to the possibilities of the medium, which allows 3D models, pictures, some sort of animations and in some extent audio and video.
The first obstacle is the little knowledge of new Media in general and of this technology in particular, that is to say, the use of the software Unity and the need to write code to open possibilities in this platform.
The challenge in general was to tell not only part of Eva’s story but also to communicate an emotional mood of her through the 3D display, and the interaction of the user with it. As well, the elaboration of a story in the mind of a user with only fragments is somehow an experimental narrative form for us, in the sense that the project is not following conventions from any particular narrative form, and actually is following the path of a forensic researcher of stories.
The Music in the Pearls
The AR Story that I designed for the pearls necklace attempts to build a subjective route through the emotions that Eva was feeling when she was abused and terrorized by her tenant, and as well, her loneliness because of her long gone husband Bernie; both of them pretty dark emotions. For this I included a dark 3D version of Eva’s house, using winter as an emotional landscape inside Eva’s psyche. Following the route from the house where Eva is lonely and scared she arrives to a representation of a place where she finds a piano and therefore, music, a long lost love. This is reinforced through a video that also is intended to be a subjective representation of Eva’s emotional mood.
It is interesting to build this little 3D sets and allow the user to explore the surroundings in the same way and archaeologist explores some old ruins, trying to find the story hidden in there. Through this model I offer some hints to the story, leaving mostly gaps to be filled by the user, and also to be completed with the stories told by the other objects. It is still early to know how the user is going to complete the stories and what I have as an author are expectations and curiosity.
Using the picture of young Eva on the side of the road during winter time, I wanted to suggest using only sound clues of what might happened that day. The sound plays ambient sounds of the road mixed with parts of two films that were very present during those days, Casablanca and Gone with the Wind. I do this by giving fragments of the soundtrack and also visual clues in a picture, so the user can invent their version with those elements. There is also a hidden face in that picture. Can that be the photographer who was also a little more in the life of Eva?
A friend asked me to help her pick up some discarded furniture from a house she had just sold for the woman who owned it, her name was Eva. As so many of ours will be the last part of Eva’s life was fraught. She couldn’t keep up with the maintenance on her home so it was falling apart. Her pension wasn’t enough and she often went to the nearby homeless shelter for food. It was there that she met a man who would eventually convince her to let him move in. His lifelong substance abuse had made him mean and he wasn’t very nice to Eva. She wanted out of her house and her nephew came to the rescue helping to facilitate her move. After a few false starts she was placed in a long-term care facility that was small enough for her to feel comfortable in. Soon after her arrival she wasn’t feeling very well and they took her to the hospital where it was determined that she needed surgery. After some complications during surgery Eva died on the operating table of a heart attack. She was 87.
When I entered her house I was saddened immediately because the condition of it was very bad with holes in the roof and some walls. As we walked around I found a few objects that had been hers that I hated to see go in the trash so I took them home with me. This consisted of a couple of paint by number paintings, a handheld mirror, a string of pearls, some tea towels, a golden handbag, a watch, a slide ruler, a porcelain spoon, a playbill from a production of St Joan, some old photos, an old slide projector and a few slides. I didn’t know when I was taking these things that they would be, but now they have been re-purposed as part of this project where we are trying to imagine moments from a possible life based on the briefest of associations through these objects.
What is an object?
There is a space between the real world and what we are capable to perceive with our senses as real. This project is about entering this space, a gap to explore the possibility of having a secondary eyes and ears, to examine what might not be visible though naked eyes and could not be heard through our ears, to imagine and to realize what a secondary set of eyes and ears could enable us to see in ordinary objects. After all, this is not such a human centered world. Objects have their own sensitivities. It is only for us to see, and to understand what Delueze calls the non-human expressivity.
Creating an AR application:
This project started with exploring the Unity game engine environment. The idea of creating an AR application was exciting enough to get us started with the Unity environment instantly. As we learned the turns and the twists in the Unity environment, our story developed, squeezed, and reshaped accordingly. Here is my take on how it shaped.
What is inside a postcard from the CNE? Who bought the card? Did she buy it herself? Did she receive it from a fiend? There is one thing for certain; a postcard from the CNE should come from the CNE.
To infuse the card with the memories of CNE, the card is overlaped with a collaged footage of CNE from the same period (1960s). The voice of it of course had to be something that sounded in ears, and tapped the feet in those days, a song by Monkees. The footages and the postcard are obtained from Canadian National Exhibition Archive. An animated ferris wheel is turning in the middle of the scene as well.
Saint Joan – a Bernard Shaw Production
What is in a theatre brochure, especially if it was stuffed under a spring mattress for decades? Where does it go back to, a day? Yes, that day, after the program, when the curtains were down one last time, and people were lined up to squeeze out through the narrow exit doors, a gaze was met with another, two set of eye met to form something more than a casual glance. At the end though, a ride on the 59 streetcar was inevitable. An hour later walking down the street and saying hi to Mrs. McKay who was walking her black labrador she realized that the moment was gone forever. So a keepsake was necessary to keep the spark, the unfulfilled dream, the unrealized possibility. What about the theater program brochure? yes, this paper back keepsake worth keeping for a lifetime.
Documentation card presents the title, names, and also a trailer about the project. The Documentation card has an interactive feature. Pressing the spiral icon leaves the program and takes the viewer to the documentation website.
When we sat down for our first group meeting, we threw around concept ideas. I tossed out a few ideas that I felt could be fun, but also had commercial potential. I suggested that we create AR fortune cookies. The Scenario would look something like this: While at a restaurant, you finish your meal and along with your bill you are given a fortune cookie of some sort. It could be in the shape of a traditional Chinese fortune cookie or it could be something completely different. Any sort of container, or vessel if you will, for the AR marker. Once the fortune cookie has been opened, a card would be revealed containing instructions for downloading a smartphone app. Perhaps the restaurant’s own app that has a “Fortune Cookie” button which launches the AR app. The fortune would appear in the form of one of one or more AR experiences including: music, spoken fortune, silent video, abstract visuals, or any number of other media. This presents the possibility for the creation of serialized content, thus engaging the patron in more than a gastronomical experience. The Fortune cookie content could be self-contained, or those at a single table, or multiple tables if the restauranteur wanted to create a broader social experience, could combine their fortunes to reveal a deeper meaning. Perhaps a bit far-fetched, but not so when one considers Jane McGonigal’s examples in Reality is Broken.
Another idea that I had was for AR event invitations. When someone receives a wedding invitation in the mail they open the envelope and read the invitation details, which include a URL to visit or an app to install on their smartphone. The invitation when viewed with a computer webcam or smartphone, depending on the technology used, would reveal video of the betrothed couple, personally inviting the recipient to their nuptials. Although group members did like both ideas, they did not fit into the Unity workflow to which we had committed. As a result, I kept my remaining list of ideas of this ilk to myself.
Talk then turned to a narrative-based project of some sort. Colleen told us the story of Eva. The initial intention for the “Eva” project was to document the life of a real person in a traditional documentary format. I wanted to blur the lines beyond fact and fiction, never allowing the viewer to know what was real and was an impressionistic rendering of reality. When Colleen told us Eva’s story I immediately thought that we should make the objects that belonged to Eva receptacles of memory. When I blurted this out to the group Colleen said that she remembered hearing a story of a castle that did just what I was suggested. Not an original idea apparently, but still a cool one. We went with it and did a bit of research into the Castle. So, in addition to the story of Eva being based in reality, so is the story of the castle. The castle in our tale is a composite of Berkeley Springs Castle in West Virginia, and Muncaster Castle in Cumbria Country in North West England.
I knew that in addition to creating game objects, I wanted to create a visceral world for the end user. My goal was to create a series of mini-vignettes that would serve as pin-hole glimpses into a life. I decided that I would only use music and soundscape that I composed myself, the only exception being “Auld lang syne” which I did not compose but this version is my own arrangement performed by Brielle Goheen on violin and me on everything else. I also wanted to only use photos and video that I shot, or at the very least, that I had found in the real world. Not on the internet. I succeeded in adhering to both of these self-imposed creatively limiting/liberating stipulations.
What follows is a brief discussion of each “vessel of memory”. Some vignettes were intended to be triggered by very different objects, but alas not everything is Vuforia-friendly.
The following is a link to an external site containing the videos that correspond to the descriptions below.
The Content of the Vessels of Memory
My mother once told me a story (Intro)
This object (picnic basket) is the user’s threshold into Eva’s world. The young Eva’s voice, transforming into that of the adult Eva, recalls a story of a haunting. This introduction presents elements (mother, superstition, water) that will be revisited over the course of the user experience.
Who is John? Who am I? (8track)
This experience is about duplicity and paradox, with references presented throughout the video clip to illustrate this theme.
My mother was superstitious. (8track back)
This object when turned over reveals the hidden side of life. The spaces beyond normal view. In this view we are privy to some of Eva’s beliefs based on stories that her mother told her. This object’s content functions as part one in a set of two. The second object reveals additional information about her superstitions and how they relate to her own life.
Eva, who’s in there with you? (25th plate)
The sound of my own heartbeat is the main soundtrack to this object. The anniversary plate is a portal into a seemingly normal life interspersed with suggestive imagery of spider webs and water. Themes of drowning in reality and going deeper into darkness are visually presented with an irregular voice (my own speaking backwards and then reversed) asking a question of Eva that could be interpreted in several ways.
The myth of Mary Magdalene (Bernie)
Every narrative needs a bit of Judeo-Christian allegory. Mary Magdalene is symbolic of Eva as follower, iconoclast, and, depending on one’s interpretation, sinful woman. Visual imagery of soup kitchens, homelessness, Toronto from the night sky, and a spider web present elements of Eva’s real story as well as threads, no pun intended, of ongoing story elements.
They speak to me (Engagement)
For this object, my own field recording of the Islamic call to prayer in Istanbul, is combined with background cafe conversations, reverse dialogue, and original music composed for guitar and violin. This piece outlines potential mental illness that may or may not have precipitated an eventual crime of passion.
What if? (Moonlight Sonata-Woman-Snowflakes)
This sheet music is a reference to the real Eva, who played piano and worked at the Royal Conservatory of Music. The sheet music target is the same piece of music played by the young Eva in one of our videos. The hidden content (behind the clock) is a continuation of “My mother was superstitious”, creating additional connections to be made between characters. This game is littered with symbolism of the mid-20th century woman, and Eva in particular.
All Ephemeral (Scan Game-Man-Sunrise)
This object is a metaphor for the transience of existence, or maybe I’m reading too much into it. Perhaps we are never anywhere, but always in between places. The real world “Scan Game” provides a series of symbols to be deciphered as a next step toward the next thing, which is in turn just after that thing and just before the next.
Times long past (White Christmas)
Auld lang syne (times long past) is used as the sonic backdrop for this experience. Christmas is always a time of looking back, never of looking of forward. Using my own family movies (I’m the little boy) combined with my own arrangement of the classic holiday song, this piece serves as a moment of nostalgia for what the passing of time inevitably takes away from us.