Tim Stutts: ITP Project Blog

Testing out the joystick interaction, i’ve realized the most satisfying graphical implementation is where there is a one-to-one relationship with the controller parameters and what is occurring on the screen, else the the user is left confused as to what they are actually manipulating visually. As an excercize I’ve constructed a grid with set orginins. The incoming values from the Arduino are applied to parameters that offset the x and y position of linked transluscent quadrangles. When the joysticks are released the orginal grid shape is reestablished, though artifacts of the previous quadrangles remain.

Watch a video of the grid in motion here.

I have yet to find a method–even through the use of arrays–that would compress the exisiting code in a way that would still makes sense visually. view the current source code here.

The following Arduino and Processing code excerpts link communication between the Lawnboard and Computer. In this basic implementation, joystick position controls hue of the squares within the grid. Right now there is a flickering issue that is a result of a fualty bitwise operation. I plan to solve this with some debugging.

Read more

M.E.L. or Malleable Electronic Lawn ( the interface formerly known as “Lawnboard” ) is looking sexy. Some recent progress made on the device includes completing 100+ wirings, almost completing the Arduino code to control the multiplexers, upgraded the Arduino to an Atmega 168 for added memory, and working with Luke DuBois on an interactive graphical representation in Max / MSP Jitter. Eventually I would like to model and control a rhombic triacontahedron, a polyhedron that has 32 vertices. Each joystick axis would control the polar coordinate of a vertex. Other tasks still to come include collaborating with Andy Doro on a secondary visualization in Java / Processing and constructing the tactile elements to fix onto the joysticks. One recommendation I’ve gotten is using thin wooden dowels surrounded by soft PVC tubing, which I will likely put together over the next couple days.

Read more

I met with Greg Shakar tonight to get the multiplexers working with the Lawnboard. Looking at the mess of wires, I’ve come to realize that I will need to horizontally mount a slightly larger breadboard onto the interface, to make space for the multiplexers and wiring. Below is the current bit of code I am using, which I will soon update to allow for 32 incoming values:

Read more

I spent many hours cutting, stripping and sodering 96 pieces wire into the the lawnboard. The vein-like arrangement of the wires are visually interesting and a bit more cluttered looking than I had anticipated! A future version should include a prefab board, where all the power and ground connections are linked, and the joystick returns are flat against the board–total wires = 0. For now though, this will be useful for testing purposes, once I hook up the analog multiplexers.

No longer just found alongside the personal computer, the mouse and its variations have invaded our mobile communication devices, portable music players, and gaming consoles. Among the interface types are the traditional mouse, track pad, tablet and track ball. All of these allow the user to control the XY position of a pointer, along with some kind of modifier in the form of buttons, scroll wheels, and pressure sensors.

In observing people using the mouse, I’ve noticed that there is always distance between the gesture communicated by the hand and resulting action on the screen. Though increasingly more fluid, the action is lost in quantization; the transfer of the analog gesture into the digital domain. The tablet, with its pressure-sensitive pen pointer, has succeeded the most in overcoming this limitation. In Photoshop for instance the tablet can be used to create lifelike brushstrokes. An alternative to capturing the human gesture via mouse interface is via scanner, though this could arguably add an interrupt to the creative flow. Artists frequently employ this technique as a beginning phase for a work, which they will later colorize and process in software.

A small handful of companies have released alternative pointer devices. While working as a sound editor in Los Angeles, I began to look into other solutions, after a friend of mine, who worked in video restoration, acquired carpal tunnel syndrome. The products I found included foot controllers, head-worn line-of-sight detectors, and a number of modifier key devices. Although accommodating to the fatigued wrist, these interfaces add even more distance from the gesture than the aforementioned devices, and serve as a last resort, rather than a solution to creative expression. As an example, using a foot pointer device to drag an automation track in a multi-track editor such as Pro Tools, would take hours. Feet just aren’t able to do what hands do best.

In conclusion, perhaps the best way to quantize gesture is to involve fingers, as well as the hands. The glove comes to mind, though this device, when manifested as an array of ribbon sensors, fails in its relevance to 3D space—the hand motions through the air, but the computer desktop remains two-dimensional. To interface with the modern computer, it makes more sense to remain in the two dimensional realm. I propose a solution, which I have already begun to develop, that is a series of vertically mounted XY sensitive fingers, connected to a board. With a traditional mouse in one hand and the new device in the other, sixteen separate instances of the two dimensional are controlled at the same time, acting as a modifier, yielding complex gestures, allowing for data to be treated separately or summed through various means.

I received the 16 CTS joystick controllers in the mail late last week, and have since affixed them to a perf board. Initially I was overly ambitious about wiring all 96 connections (2 ground, 2 power, 2 arduino returns for each!) in one shot, but at the recommendation of Carlyn Maw, I have decided to keep things simple and use only 3 of the joysticks for now. The Arduino board has only 6 analog inputs, so to use more joysticks will require multiplexers. The next course of action is to experiment with materials for the grass blades, and also determine the best use of the interface, perhaps to control a brush tool in a graphics application or audio plug-in parameters in Ableton Live or sixteen simultaneous games of Pac Man or for building 3-D landscapes in Terragen.

I wonder what Cory would think of this?

Lawnboard is an interface comprised of 16 CTS Series 252 miniature joystick controllers with synthetic grass blades affixed to each; a total of 32 parameters along the X and Y axis. The user interacts with the device by waving their hand through the lawn, triggering small movements of the various joysticks, connected to an Arduino board, which in turn controls computer software.

My inspiration for Lawnboard arrived through my exposure to the joystick, and its function as a fluid 2D controller, as well as two projects at the 2006 ITP Winter Show, that involved patterns of growth and communication with plant life. Lawnboard uses a modeled plant structure as the sensory apparatus.

Specs on CTS Series 252.

Josh Knowles “The Botanical Garden.”

Robert Faludi, Kate Hartman, Kati London, Rebecca Bray “Botanicalls.”