Tim Stutts: ITP Project Blog

HeartPeggio is an apparatus worn on the body that measures heart rate and generates a sequence of tones. White noise “ticks” occur at regular intervals to determine a goal rate. The user relies on biofeedback or exercise to align their “active rate” with the goal rate.

When I originally approached Raphael Zollinger about working on a final project for Advanced Microcontrollers, we were interested in the idea of using information from body and having that data represented as sound. We did some research and discovered the work of artist / medical technologist, Atau Tanaka, who had relied on neural information and various body sensors to trigger and manipulate sound. This led us to a discussion about fitness equipment, particular treadmill devices that track the heart rate. We decided to design a piece of hardware that coaxes the user into a “Zen” state via positive incentives for a more relaxed heart rate, as opposed to the reward mechanisms for pumping as hard as you can, found in the modern gymnasium.

As far as the actual sonic content, it was initially decided by myself, to use two low tones, tuned within 10 Hertz of each other, and have the frequency of these tones be function of goal—or ideal—rate and active—or current—rate. As the user’s heart approaches a state of calm, the two tones would align, resulting in an out-of-tune fluctuating effect that can only be resolved through a concentrated biofeedback that would move the tones to unison—an idea influenced by the drone in Indian Music.

In theory this worked out well, but when I began experimenting in Max/MSP, Raphael pointed out that it was hard to distinguish separate tones, let alone difficult for the untrained ear to determine which tone represented the goal rate and active rate.

I then turned to the idea of phasing, in reference to a method developed by composers Steve Reich and Terry Riley, where similar or identical phrases of music or sound are played at the same time, with one moving slightly faster than the other. The resulting effect is unsettling, however meditative, once the ear has adjusted to this non-classical division of tempo. The effect can be accomplished with either live musicians—in the case of Reich’s “Drumming”—or tape—Reich’s “Come Out.”

I conducted similar experiments in Max/MSP, this time having the goal and active rates triggering melodic arpeggios. Initially both phrases were in the same harmonic range, but again I faced the problem of the ear being able to distinguish the significance of each. I tried placing the two passages an octave apart, changing the timbre of each, and even having the top passage move twice the tempo of the bottom. Though pleasing to the ear, none of these approaches conveyed clearly distinguishable goal and active rates. It wasn’t until later that I determined that the most effective approach was having the goal rate represented with a sharp envelope of white noise and active rate, a four note arpeggio, with clearly distinguished tones created through FM synthesis.

Throughout this whole process, Raphael had been researching heart rate sensors. We found some promising experiments that had been conducted at ITP, one that even used a clothespin combined with an IR sensor that measured light passed through the finger; developed by Tom Igoe. In the end, due to reliability issues, we determined to actually buy a sensor and hack it. We traveled to a couple medical equipment stores in town, and settled on a Polar sensor at Paragon Sports. Through experiments conducted with Gary Schoeber, we determined that the radio frequency coming from the sensor via wireless transmitter to received, was much too low to work with, without having to construct a specialized coil that receive the frequency. We returned the polar sensor.

Meanwhile I entertained the idea of being able to run the project on an iPod running Linux and Pure-Data software—a visual programming language similar to Max—but my iPod didn’t support the package and crashed each time I tried to load it. We decided that it would be best use our own hardware for obtaining data, in conjunction with a laptop computer to actually generate the sound.

While Raphael began the process of programming the Arduino, and designing and building the remaining hardware, I got to work on a Max patch that would receive a signal from the Arduino using the serial object. When we ran into issues having the Arduino send a string of integers to Max, due to limited byte size, I proposed having all of the information contained in a single integer that would range from 0-255. The Steps below explain the conversion method I devised, that involves cooperation on the part of both Arduino and Max. I essentially take variables for goal rate, active rate, goal rate on/off, and all rates on/off can confine them to different ranges.

- Arduino generates variable for goal rate and active rate and converts them to delays milliseconds and then assigns them to variables.

- The maximum and minimum useful value for goal rate and active rate are 300 – 1500 milliseconds (roughly 40 – 200 bpm).

- Goal rate variable is divided by 10 and then subtracted by – 29. This confines it to a range of 1-121.

- Active rate variable is divided by 10 and then added to 100. This confines it to a range of 130-250.

- Active rates that exceed the allowable rates—falling into the upper extremity of 122-129 or lower extremity of 251-255. This rate doesn’t normally occur unless your finger leaves the sensor, in which case the active rate shoots way up, and the Max receives a message to stop both goal rate and active rate sounds.

- A message of 0 is sent periodically by default through the serial object, which mutes the goal rate. Once this button is pressed the goal rate is active and heard by the user.

The resulting confinement of data helped make HeartPeggio a success! Please see the attached Max patch below.