Sound Interactions

About

Who am I? I am an interaction designer with a background in sound design for film, games and installations. My ongoing project has been to blend the best practices of both domains to create immersive experiences for the user / audience. I'm equally at home with conceptualization, usability testing, wireframing, and spec writing, as I am building prototypes from code--whatever it takes to give the final product that extra push.

How did I get here? I had my creative beginnings as a student of drawing and ceramics. In the early 90s, I took up the electric bass and studied into high school, performing in jazz band, and taking classical lessons on the upright. I was eventually admitted into the Music Composition program at University of Miami, Florida, where I studied classical composition and jazz. In 2001 I made my way out west to attend California Institute of the Arts program in Music Technology. There I studied film sound design, synthesis, and interactive performance, and began playing with Pure Data and Max / MSP software. In 2001 I co-founding electronic music group Cutetheory, and began making tracks under the alias Lordx. After graduating Calarts in 2003, I moved to Hollywood, where I worked as a sound editor on films, games, and commercials, while adding the sound design and programming layer to several art installations and performances. In the Spring of 2006 I taught and authored the syllabus for the Digital Audio Editing course at California Design College. Shortly afterwards I moved from New York and worked at Apple Store Fifth Avenue as a Creative instructor, teaching people ways to use computer applications on the Mac. I was concurrently enrolled at New York University Masters program in Interactive Telecommunications. In addition to coursework, I interned at Eyebeam Gallery and also Phreesia, a startup that designs touchscreen patient intake devices for clinics. In Summer of 2008 I made way out to San Francisco, where I took a UI Designer position at Intuit, designing software offerings for the small business ecosystem. In my spare time, I enjoy making music, traveling, and playing with Java Processing.

A wireframe for a sailing game on the iPhone that I'm currently designing as a side project. The player controls the force and direction of the wind to guide the boat to the finish line.

Interaction Design

Click any of the above image links to load content into this space.

Medical Touchscreen
a device for gathering patient data in clinics

The Goal
Administer and conduct a usability test to assess and later solve problem points during the patient interview process on the touchscreen device.

My Process
Conducted a usability test. Established a spectrum of attitudes patients experience during an interview--ranging from resistant to enthusiastic. I went through video of the tests, then rated the subjects's attidude towards each screen. After establishing the problem points, I designed some screens to try to make the process smoother, relying on visuals instead of text wherever possible, since this seemed to be the going trend with less problematic screens.

What I Learned
Mood is a very difficult things to capture. It's entirely subjective, and can be just as much a product of the environment, as the patient's experience on the actual pad. However, when captured accurately, it proved to be very helpful in indentifying problems and solutions proposed by subjects themselves.

This is a chart I designed that plots a patient's emotional trajectory during a typical interview on the touhcscreen device. To arrive at this data visualization, I watched videos of the usability test, and noted the reaction of the subjects each step of the way. Click on the image for a full-sized view.

A screen I designed for one of the problematic areas in the interview process--identifying where it hurts. Previously there had been an alpabetical list of conditions, but this proved difficult, since many of the issues weren't commonly known by name. To remedy the situation, I created a body figure with buttons. A patient would choose a region, drill into that region to pick a sub-region and so on, until pinpointing their chief complaint. Click on the image for a full-sized view.

Grass Type
application for typing text on to a virtual grass lawn

The Goal
Build an application for users to type text onto grass using C and OpenGL programming language.

My Process
Grass Type was a collaboration between myself and designer Che-Wei Wang. I focused on designing the individual grass blade, so that it had a convincing reponse to light and movement, and then built a model in code. Che-Wei then populated the screen with these blades and worked out a method to have individual blades bend to immitate image files containing text. The completed priece was shown at the ITP Winter Show in 2007, as downward projection onto a canvas. Users could then type text in via keyboard on a podium and interact with the grass using a mouse. The piece recieved Honorable Mention in the 2008 Adobe Design Achievement Awards, and images from it were sold off to a design studio in Spain to be used in an umbrella product.

What I Learned
It's possible to conserve computational resources by creating an illusion. The graphics, though they appear 3D, are actually only 2D, thus enabling a much smoother response that wouldn't have normally been possible.

This image shows the life-like look of the grass. 5000 blades of varying height and sway, react to the mouse, as if blown by the wind. Faint remnants of text remain. Click on the image for a full-sized view.

Download a Mac version of the Grass Type Application for desktop. Use your keyboard and mouse to interact with it:

View a sample of annotated source code.

A user interacting with the piece at the ITP Winter Show, types the letters "I X F," then moves to the mouse about. Click on the image for a full-sized view.

Malleable Electronic Lawn
a hyper-tangible, gestural interface for 3D graphics

The Goal
Design an interface for controlling multiple parameters of a 3D graphics application through complex hand gestures.

My Process
I began by thinking of ways fingers could be used as input. I became interested in spring-loaded joystick potentiometers because they could be pushed in any direction and would return to their upright position on release. I ordered 16, secured them to a perf board, then hooked them up to some multiplexers and an Arduino microprocessor. From there I wrote a Processing applet to monitor my gestures on the computer. Once I was satisfied with the responsiveness I was seeing, I began the construction of a better plexiglass housing for the device, as well as comfortable tactile fingers of wooden dowel and rubber tubing. At this point I'd spent so much time on construction, that I ran out of time to create the on-screen counterpart. I ended up connecting M.E.L. to Zach Layton's Sonic Topology piece, setting each joystick X and Y axis as a controller for different parameters in his beautiful 3D projection. M.E.L. and Sonic Topology debuted together at the ITP Spring Show 2007 and had a second showing at the Sony Wonder Lab.

What I Learned
In the future I would like to write a tutorial to teach people how to familiarize themselves with the device, as the 32 channels of input can feel like sensory overload to some. I'm also interested in exploring other applications for the interface, including using it as a surround-sound panner for film sound post-production, or having it walk a 3D marionette across the screen.

Attaching the joysticks onto the perf board, each connection was carefully numbered. If I ever mass-produce M.E.L., I will design and produce the board in advance, thus preventing me from having to connect over 200 soder points by hand. Click on any image for a full-sized view.

View a video of M.E.L. at the ITP Spring Show and Sony Wonder Lab:

Above: M.E.L. in its completed state. Below: Zach Layton controlling Sonic Topology with M.E.L. at Sony Wonder Lab.

Lumen
stairwell installation featuring interactive light and sound

The Goal
Contracted through artists Electroland to program an interactive multi-channel playback system with custom sound design for The Cooper-Hewitt Design Museum International Design Triennial in 2006.

My Process
I meet with Electroland, who had already worked out the lighting aspect of the piece, and we had discussions about what kind of sounds they were looking for. Having sounds that matched the soft lights was key. I began editing samples together on the computer, often while on site, so we could test them out in the space immediately. At the same time I began programming a Max patch that would read in values from the motion detection component of the installation and selectively trigger, place, and move the sounds through 7 speakers mounted behind the installation. The effect of interactive sound, coupled with light, was highly immersive for museum-goers.

What I Learned
Effective User Interace Design was really important for Electroland to audition different sounds in the software, as well as set volume levels and custom equalization.

I built the UI directly into the Max patch, providing clear instructions to the artists, as well as a number of components to recall custom settings. The important "guts" of the program were left exposed, so that they could be changed with little effort. Click on the image for a full-sized view.

Watch a video of Lumen at the Cooper-Hewitt Museum. Can you figure out what kind of user interactions cause certain sounds to be triggered?

Lights pulse in response to museum-goers colliding and moving on the stairs.

Conveyor Player
associative matching game for special needs children

The Goal
Designed and developed a game for children at Seton Hospital, as part of an Inclusive Games course at NYU.

My Process
I partnered up with Charlie Miller at ITP, to create a game that with simplest user interface possible. Conveyor Player is a game of associative matching. Players choose what types of objects they want to work with--animals, letters, etc.--and those objects make their way down a conveyor belt in a factory. The objects are then guided into the appropriate bin by the player, who changes the direction of a conveyor belt, by touching a computer screen, pressing buttons, or clicking either side of the screen with the mouse. I programmed the entire game in Java Processing, while Charlie served as project manager. We made frequent trips to the hospital to test application, and refined the design each time, until it matched the user need.

What I Learned
A reward mechanism is important in any game. Conveyor player takes this to the extreme, by counting matches, displaying fireworks with each match, displaying a text cheer with accompanying voiceover, and even having the bins nod or shake based on outcome. The kids loved it.

The way the game is setup in the children's classroom. The computer is outfitted with a touch-screen and a button to control direction. The UI is an earlier start page--we later added a hints feature, as well as a belt animation to show players what a speed setting would actually look like.

Download the game or watch a video of it here. (Once at the site, scroll to the bottom of the page to get to Conveyor Player.)

Above: Player chooses two categories--art and transportation. Below: Player is rewarded for a match.

Livelyhood
nieghborhood mapping application for children

The Goal
There are plenty of web-based applications for situating oneself a onto a predefined map. Livelyhood is a web application, where children explore preception of community, by drawing a map of their neighborhood, composed of the static structures and moving bodies that are a part of their day to day experience. Once drawn an object will come to life, taking on a unique set of behaviors; an archetype of it's real-world equivalent. Cars drive along roads, factories produce smoke, plants appear bushy, etc. The maps serve as location-based blogs that are continually updated by the child, with the intent of giving them a voice in urban planning.

My Process
Intially I constructed wireframes for a UI reminiscent of SimCity, a computer game that I loved as a child. Eventually I gathered drawings from students of their neighborhoods and did a comparative study to determine a common set of objects, how they divide space, and perceive distance. I quickly scratched the SimCity idea and went back to the drawing board. I then spent a many hours creating a sophisticated map-making application in Java Processing. Finally I tested the application with a classroom of kids in the Bronx and featured the results in my thesis presentation at ITP.

What I Learned
When map-making, drawing is much more expressive than dragging and dropping objects onto a screen. Beyond the grid is where an individual's voice truly comes out.

An early wireframe has users placing objects on a grid to build their neighborhood, and then notating those objects with floating tags. Around this time I was still considering using a cellphone as the platform, as is evident with the numbered on-screen buttons that correspond to buttons on the phone. I left this concept after determining that few younger people were in possession of smart phones.

Watch the Livelyhood video of my ITP Thesis presenation.

The above user sketch contibuted to the prototype below.

Push Music
music composition visualization in 3D space

The Goal
Creating an exciting way of displaying and visually interacting with music compositions. I've often found patterns in music to be pleasant to the ear, but difficult to appreciate on the printed staff or editing interface within music composition software. I wanted these structures to speak in a new way.

My Process
I began by creating a Max patch to extract MIDI-numeric data controlling time, pitch, etc.--from compositions on the computer. I then feed this in a data in realtime over to a program in C and OpenGL within OpenFrameworks, so that they music could be played in one program and visualized in another. From here on out I partnered with designer, Rui Pereira, to program visual look and feel of the application. We decided on a flower-like visualization, where time is the Z axis, and individual instruments fan out into petals, whose height indicates pitch. While the piece is playing back, the user is free to navigate its structure in 3D space with a mouse. The final project debuted at the 2007 ITP Winter Show.

What I Learned
Getting things to work smoothly accounted for 95% of the project, while the final strides in visual design happened in a few hours of the night. Never underestimate the time it will take to simply get an application functioning.

Piano-roll style is a common, unexciting way of viewing music information. Pitch is the Y axis, and time is the X axis.

Watch a video of Push Music in operation, featuring music data from my own composition, "Series."

Users watch a composition unfold at the ITP Winter Show.

Blogs

Error: Embedded data could not be displayed. Click <a href="http://timstutts.tumblr.com">here</a> to view it externally.	Subscribe to Sound Interactions Blog! Also check out... Designing Possibilities I'm a cofounder and regular contributor to this effort. Open Processing Portal Some interactive applets I've made in Java Processing. ITP Project Blog Things I made while I was a student at NYU (archive only).

CV

Profile

I am a tech-savvy interaction designer seeking a full-time or contract position, where my role would involve a combination of user experience, visual design and front-end engineering, to deliver cutting-edge software for a mobile or desktop offering.

Experience

User Interaction engineer, betteroffline | sunnyvale, CA | MARCH - SEPTEMBER 2009

- Overseeing interaction design, visual design, and user experience for BetterOffLine?s website and offering, with web-based and mobile applications providing trade-show search and analytics capability to be released in the coming months.

- Conducting surveys, usability studies, and generating personas to gain a better understanding of our core users--attendees and exhibitors at events.

- Directed and edited proof-of-concept video for the product.

- Creating graphics assets and mockups for UI elements within Fireworks, Photoshop, and Illustrator.

- Programming the front-end using HTML, CSS, and JavaScript, using Dreamweaver

- Designing and developing heat map algorithms and visualizations in Java.

USER INTERACTION DESIGNER, INTUIT INC | MOUNTAIN VIEW, CA | JUNE 2008 - MARCH 2009

- Designing payroll applications suited for the small business ecosystem, within Gemini, Intuit Online Payroll, and Service Assisted Portal offerings.

- Spec-writing and building prototype in HTML, Flex, and Java.

- Conducting usability studies to test different screens with potential customers.

USABILITY INTERN, PHREESIA | NEW YORK, NY | OCTOBER 2007 - MarCH 2008

- Created prototype graphical user interfaces for a touch-screen device used to replace the patient clipboard in clinics, using Adobe Illustrator, HTML and Java.

- Administered, gathered and analyzed data from usability studies, producing emotional-trajectory graphs in Excel, tying into video segments.

Creative, apple store fifth avenue | NEW YORK, NY | OCTOBER 2006 - October 2007

- Instructed one-on-one lessons on web design, video editing, and music production.

- Presented workshops to a live audience on iTunes, iPhone, and other Apple offerings.

Course Instructor, art institute | los ANgeles, CA | FEBRUARY 2006 - JUNE 2006

- Taught Digital Audio Editing course to students in the Interactive Media Design program--course combined lessons in sound theory and practice in Pro Tools software.

- Authored syllabus for Sound Design course.

FREELANCE SOUND DESIGNER, VARIOUS STUDIOS | LOS ANGELES, CA | MAY 2003 - JUNE 2006

- Self-employed audio editor on numerous projects for film, television, games, radio, theater, museum, audio books and children?s toys, working at such studios as Technicolor Interactive, Warner Brothers, Outlaw Sound, and Video Box.

- Garnered over ten IMDb credits, with projects including ?Lottery of the Sea? feature documentary by Allan Sekula, ?Jade Empire? game for Xbox, and ?Justice League Heroes? for Playstation 2.

Education

- New York University, NYC - Masters of Professional Studies, Interactive Telecommunications (ITP), ?08

- California Institute of the Arts, Valencia, CA - Bachelors of Fine Arts in Music Technology, ?03

Skills

- Experience in evolving prototypes into products, usability testing, wire-framing and spec-writing.

- Knowledge of HTML, CSS, JavaScript (JQuery), Flex Builder, Subversions, and Adobe Creative Suite.

- Rapid prototyping skills in Java (Processing) and C with Open GL (OpenFrameworks).

- Understanding of physical computing with basic analog and digital electronics.

- Background in film post-production using Pro Tools and Final Cut software.

- Expertise in Max/MSP programming for museum installations.