Interaction programming

'Fragments of Privacy'
Installation with Masha Starec,
at Bartók, Ludwigs Museum, Budapest.
Oct 08, 2016 to Jan 28, 2017

budapest1 budapest2 budapest3 budapest4

Embodiment - Dance in the Interactive Multimedia Environment
workshop and performance 2014

This workshop series took place in early summer of 2014, when I had opportunity to work with Kiwi Chan, local Hong Kong dancer, and her performance group HerStory Polygon. Over ten days we experimented with motion tracking, camera interaction, semiautonomous lighting to create a dance performance in an Ambisonics 3d surround sound field. This video shares one element of this performance, in which camera input to MAX/MSP directs the playback of audio within the 3d soundspace, and a motor lamp wanders the field, also triggering sounds with it’s movement. At the end, audience members were invited to participate in and investigate the dynamic sound field.

Workshop Flyer (pdf)


Another experiment with embodiment, Brain wave EEG device used to control motor lamp (meditation level mapped to lamp rotation speed) with dancer Ero.

<# some text #>

Brain Computer Interface -> Max/MSP -> Arduino DMX -> Motorized LED Lamp Fixture


Participants also toyed with the necomimi, an original Japanese cosplay accessory, cat ears move depending on EEG input.

Check out my cool robot!

Actually this is a Braitenberg vehicle, built with Arduino. It is a simple reactive intelligence which mimics oceanic protozoa. Photovoltaic sensors control motors which propel the vehicle, or creature, towards or away from light, much as some plankton navigate the water column, rising in the day to feed on, or absorb sunlight.


When this little guy is wandering around in search of light, or more sadly, in search of darkness, it's hard not to think of him, or her, as him or her. It's very engaging.

I have done quite a fewdevelopment projects with arduino controllers. I experiment with a number of kits such as wav players, mini amplifiers, lights and sensors.

My favorites are ultrasound distance sensors,


and the xbee wireless modules.


Here's a project which takes the microcontroller paradigm several steps further, an Arduino DMX lighting controller, connected over wifi and controlling Red Green Blue and Alpha dimmers using an ipodTouch and TouchOSC. Very cool, but is it any better than the old Disco 512 dimmer?


Interaction Programming is not just making things which interact with the world, or giving audience some input into a media object. The interaction in my own work is in fact not a way for audience or viewer to engage directly with my work, I dont often give you buttons to press or configure my media art to respond to you. I program interactivity as a tool with which to engage with my audio and visual media, a utility for creating and navigating an installation or performance space.


One of my main interaction programming elements is ambisonics, using PureData or Max/Msp. With this technology I create three-dimensional sound fields, a performance and installation environment, where I can mix and spatialize sound units, or actors, feeding six or eight monitors and two subwoofers, through hardware and software effects and mixers.

Here is a photo of the setup during testing at Penghao Theatre in Beijing. An ambisonic sound field is interactive in its very existence as the physicality of the sound elements, and the unity of their presentation and absorbtion effect the listener in such a profound way.

ambisonics_beijing1 ambisonics_beijing2

Linux computer running Ubuntu and Pure Data, Presonus 10-channel Firewire Audio interface, Microphone input, external audio and hard-drive based audio, spatialization via Kaoss Pad 3 touchpad controller, 8 active monitors and Subwoofer, plus one media guy!


digital ink - max/msp ink painting animation machine. This is an interaction system which creates, loops and mixes stop-motion animation while I paint.

setup1 digital_ink_patch

Here is the animation machine in a Pure Data iteration. Click on the thumbnails to see the results. (sorry flash only on this one)

Hong Kong Virtual 香港虚实

Visit the site

I built this Flash ActionScript 2.0 virtual environment back in the day when Flash development seemed like the future. Programming random crowds travelling on virtual trains is no simple thing. And we want the trains to run on time... was my first foray into interaction programming, and I created this mock-up of Berlin's East Station in Flash. The project got out of hand, as i was a single developer doing the whole thing, and while I could create the panorama, in the end the virtual world is quite empty. I simply could not animate enough stuff to make it lively.

ostbahnhoflogo is an experimental animated art world begun in 2003 and offline now for many years. Looking at it now, I smile. Oh, were we all not once, so young and foolish?