Open Kinect Library for Windows!

Finally! I just discovered this. It was posted about a month ago but is worth a look.

http://forum.processing.org/topic/kinect-library-dlibs-freenect

Assignment 3 - test 1 - mouse interaction

View my test sketch at openProcessing.org.

This a low sound quality (web download friendly) version of my mouse interaction testing sketch I have up and running. Its pretty self explanatory. Each section in the screen represents a position and the mouse replicates a person walking in and out of these positions.

My next step is to start using OpenKinect to get 'interfaceless' interaction!

View my test sketch at openProcessing.org.

Assignment 3 - Coding...

Currently I am still trying to get my head around how my sketch will work. How will each sound lead on to the next? What will trigger the different sounds? How will the participant be able to explore the interactive work? Below is my first attempt at Object Oriented Programming in an attempt to sort out the order in which all the components of the work will fit together.

My next order of business is to write out each of the functions listed above and to get a mouse interaction demo up and running. See you next time!

Assignment 3 - Sound Ideas

This is what I've been working on in the sound department. Enjoy!

SOMA2607 - InteractivityTrack 02 by Timothy Watt

Assignment 3 - The 'Full Body DJ'

Last week I gave a presentation explaining my idea for my final project and this is more or less a transcript. Enjoy!

The 'Full Body DJ'

Full body DJ-ing is about 2 things. Firstly it is about ‘body music’. What I mean by this is interacting with a sample audio track without an immediate interface. This means navigating, effecting, triggering additional tracks and adding your own samples to create a mix. Secondly it is about ‘embracing the glitch’. Delays, pops, and other imperfect aspects of digital audio manipulation will be embraced. Their inherent faults will be used for musical aesthetics.

My first inspiration is ‘Subcycle’: a work developed by a guy called Christian Bannister. In my opinion, with his 'Minority Report' style, multi-touch interface, he has taken interactive electronic music to a whole other level. He is able to navigate, reverse, bit-slice and granulate sample tracks in real time. His dynamically generated visuals are also very impressive.

ttp://www.creativeapplications.net/processing/multi-touch-sound-storm-processing-sound/

My other major inspiration is a work called ‘Twill’ by Matt Pearson or ‘zenbullets’ as he likes to be known. It one of 100 works in his 100 Abandoned Artworks series which can be found on his website. I really like how this work looks. The simple black and white is very effective and the flowing form in the middle is mesmerizing.

http://www.creativeapplications.net/processing/100-abandoned-artworks-processing/

I really like the idea of a dynamic waveform that follows the participant’s (similar to Petra’s digital calligraphy) and I also really like the look of the black and white forms from Matt Pearson’s series so I might aim for something along the lines of this for the visualisation.

The audio that I am working on is a short piece of ambient electronic music. It will be around 20seconds in length and is based around the one key so that the different tracks can start at different times but still work together. Once the piece is built up by the participants interaction I want it to sound rhythmical, looped and glitchy.

The participant will interact with the program by their movements in front of an Xbox Kinect. Changes in the soundscape will be trigger by four parameters:

1.       Presence (background subtraction) – for example to start a rhythm cycle.

2.       Position (frame differencing) – to control audio effects, scrubbing, etc.

3.       Depth (infrared via Xbox Kinect) – visuals

4.       Volume (dB level) – don’t really know if I could do it as of yet but it’s more interactivity that could be utilised.

My plan from now until the project is due in four weeks is as follows:

-          For the rest of this week (and maybe next) I will finalise the projects sound library. This will really just involve deconstructing a segment from one of my current tracks and formatting them into 20 second intervals.

-          Starting next week I will divide my time between focus on the sketch’s functionality and the visuals. As for functionality, I currently have a simple mouse interaction prototype in the works so the next step will be incorporating data collected by the Xbox Kinect into this interaction. After that the obvious testing and tweaking.

-          As for visualisation, I plan to borrow code from Petra’s ‘Simple Digital Calligraphy’ and have a dynamic waveform that follows the movement of the participant. I haven’t started this and I’m guessing that a lot of experimentation is in order.

Assignment 2 - Research

These are a whole lot of works that I found inspirational; either visually or conceptually.

!!!PHENOMENAL!!!
http://www.creativeapplications.net/processing/multi-touch-sound-storm-processing-sound/ - Christian Bannister...this guy is my hero. With his 'Minority Report' style interface, he has taken interactive electronic music to a whole other level. Through this multi-touch display he is able to navigate, reverse, bit-slice and granulate sample tracks in real time. I would love to get my hands on one of these machines!

For Xbox Kinect
http://www.shiffman.net/2010/11/14/kinect-and-processing/ - Author of openKinect

Creative Applications Network
http://www.creativeapplications.net/ - Great resource for viewing new interactive works.

Time and Motion Study - John Tonkin
http://vimeo.com/5834154
http://vimeo.com/5834255 - 3D representation recorded video capture - Brilliant!

Great sound visualisations
http://www.youtube.com/watch?v=4rI3LGyD-3o - Painting with sound
http://www.youtube.com/watch?v=iF-2Fyie8z4 - 3D Energy waves
http://www.creativeapplications.net/processing/100-abandoned-artworks-processing/ - Beautiful aesthetic
http://www.creativeapplications.net/iphone/cloud-data-iphone-ipad/ - Data cloud!
http://www.creativeapplications.net/maxmsp/visual-music-collaborative-events-results/ - Great dynamic visualisations
http://www.creativeapplications.net/processing/nanokey-experiments-processing-sound/ - Relatively simple but very effective

Presentation
http://www.creativeapplications.net/processing/radius-music-processing/ - Nice use of light and shadow
http://www.creativeapplications.net/processing/rhifid-speakers-processing/ - Monitors, wires and space! Great presentation of a not-so-amazing concept ;)

Interactivity
http://www.creativeapplications.net/mac/gestural-music-sequencer-processing-sound/ - Amazing method of interacting using different coloured lights.

Animation and editing of video to sound
http://vimeo.com/3173439
http://vimeo.com/12771615

Assignment 2 - Ideas...

Idea 1 – Group Speak

A program that runs a line of text. A quote. The user reads the text hopefully at pace. It is then played back to the user as a composite with other people’s responses. Chopped up and changed around.

Idea 2 – Changed Perception of Reality

An augmented reality program similar to ‘Inception; program on iphone. Plays music and user wears a headset with a microphone near their mouth. Audio input is heavily layered with reverb while trippy visuals are found from the internet from around where they are. Purpose, changing the perception of the users immediate reality.

Idea 3 – Interactive Cloud of Sound

A simple interface with an audio waveform at the centre. Using a composite of soundscape’s, voices, and current sounds a ‘sound cloud’ is created.

When nobody is present, the program is relatively dormant with random outbursts of sound to trigger interest from passers-by. When the user approaches the screen they notice that they are able to affect this cloud. For example, their vertical height compared to the screen affects the high-pass frequency while the speed of their movements dictates the volume, etc, etc. Any sound that the user makes over a certain volume is then added to this cloud and is either immediately played back or saved for later.