Hear to Create

Augmented Reality Creative Journey

Hear to Create

TL:DR

Hear to Create is an AR experience that brings the internal creative journey of a music creator to light in a way that the general audience can understand and enjoy. It shows how the scenery, experiences, sound and happening of Washington Square Park sparks creativity. It prompts users to get out of their bedrooms and explore the outdoors.

To do this, the creative process itself is represented through a “hear and replace” mechanism that breaks down the creative process for modern attention spans.

Synopsis

It’s a Sunday afternoon. You woke up not long ago because you were up last night trying to figure out the right sound for an instrument in this new music project you started. You realized your dorm smells a little stuffy so you went ahead and opened the window.

You hear sounds coming from the Washington Square Park that is so close to your dorm but you haven’t been to for a while. You decided to go for a walk. And this creative journey starts here.

You step into the park. You hear crickets hopping in the grass, birds chirping, and all types of other sounds. You suddenly feel inspired and you decide to focus on one sound at a time. As you walk you started to turn these sounds from the park into music in your head. The music gets fuller and more complete the more you walk.
By the time you exit the park. You’ve finished this song in your head that you’ve were struggling to feel inspired back in the dorm.

User experience

The user will experience this story in a primarily auditory and mixed reality format. The graphical elements are mostly functional as they fade into the background and only serve as call-to-actions.

context aware media final presentation 6.010.jpeg

Characters

This is a single-character, first-person point-of-view story. The user takes on the role of a college music producer who makes music from his or her dorm, and decided to go for a walk to the park outside.

Story map

context aware media final presentation 1.001.jpeg

User starts on the position of the blue arrows. Each numbered yellow block represents:

An environmental sound and
A trigger to turn that environmental sound as an element in the music the creator is coming up in his or her head.

Once user reach each location as labeled in yellow, a “use this as inspiration” button appears, allowing the user to turn environmental sound into a new layer to the music loop that is playing.

At position 1, the user gets to experience the sound of crickets, and that subsequently turns into the first element in this music in the making progress.

At position 2, the user hears birds chirping. As the user clicks “use this as inspiration,” there is an additional layer added to the music.

At position 3, the user hears the sound of a water fountain, which is realistically accurate as that is where a water fountain is located inside the Washington Square Park. This environmental sound element inspires the bass note of the music.

At position 4, the user notices the sound of skateboards, and how they make distinct clicks as they hit the ground or crash into any surface. This then becomes the kick of the music playing.

At position 5, the environmental sound mimics children playing. This sound inspires the creator to add lively finishing touches to the music as he walks out of the park.