Open Kinect Library for Windows!

Finally! I just discovered this. It was posted about a month ago but is worth a look.

http://forum.processing.org/topic/kinect-library-dlibs-freenect

Assignment 3 - test 1 - mouse interaction

View my test sketch at openProcessing.org.

This a low sound quality (web download friendly) version of my mouse interaction testing sketch I have up and running. Its pretty self explanatory. Each section in the screen represents a position and the mouse replicates a person walking in and out of these positions.

My next step is to start using OpenKinect to get 'interfaceless' interaction!

View my test sketch at openProcessing.org.

Assignment 3 - Coding...

Currently I am still trying to get my head around how my sketch will work. How will each sound lead on to the next? What will trigger the different sounds? How will the participant be able to explore the interactive work? Below is my first attempt at Object Oriented Programming in an attempt to sort out the order in which all the components of the work will fit together.

My next order of business is to write out each of the functions listed above and to get a mouse interaction demo up and running. See you next time!

Assignment 3 - Sound Ideas

This is what I've been working on in the sound department. Enjoy!

SOMA2607 - InteractivityTrack 02 by Timothy Watt

Assignment 3 - The 'Full Body DJ'

Last week I gave a presentation explaining my idea for my final project and this is more or less a transcript. Enjoy!

The 'Full Body DJ'

Full body DJ-ing is about 2 things. Firstly it is about ‘body music’. What I mean by this is interacting with a sample audio track without an immediate interface. This means navigating, effecting, triggering additional tracks and adding your own samples to create a mix. Secondly it is about ‘embracing the glitch’. Delays, pops, and other imperfect aspects of digital audio manipulation will be embraced. Their inherent faults will be used for musical aesthetics.

My first inspiration is ‘Subcycle’: a work developed by a guy called Christian Bannister. In my opinion, with his 'Minority Report' style, multi-touch interface, he has taken interactive electronic music to a whole other level. He is able to navigate, reverse, bit-slice and granulate sample tracks in real time. His dynamically generated visuals are also very impressive.

ttp://www.creativeapplications.net/processing/multi-touch-sound-storm-processing-sound/

My other major inspiration is a work called ‘Twill’ by Matt Pearson or ‘zenbullets’ as he likes to be known. It one of 100 works in his 100 Abandoned Artworks series which can be found on his website. I really like how this work looks. The simple black and white is very effective and the flowing form in the middle is mesmerizing.

http://www.creativeapplications.net/processing/100-abandoned-artworks-processing/

I really like the idea of a dynamic waveform that follows the participant’s (similar to Petra’s digital calligraphy) and I also really like the look of the black and white forms from Matt Pearson’s series so I might aim for something along the lines of this for the visualisation.

The audio that I am working on is a short piece of ambient electronic music. It will be around 20seconds in length and is based around the one key so that the different tracks can start at different times but still work together. Once the piece is built up by the participants interaction I want it to sound rhythmical, looped and glitchy.

The participant will interact with the program by their movements in front of an Xbox Kinect. Changes in the soundscape will be trigger by four parameters:

1.       Presence (background subtraction) – for example to start a rhythm cycle.

2.       Position (frame differencing) – to control audio effects, scrubbing, etc.

3.       Depth (infrared via Xbox Kinect) – visuals

4.       Volume (dB level) – don’t really know if I could do it as of yet but it’s more interactivity that could be utilised.

My plan from now until the project is due in four weeks is as follows:

-          For the rest of this week (and maybe next) I will finalise the projects sound library. This will really just involve deconstructing a segment from one of my current tracks and formatting them into 20 second intervals.

-          Starting next week I will divide my time between focus on the sketch’s functionality and the visuals. As for functionality, I currently have a simple mouse interaction prototype in the works so the next step will be incorporating data collected by the Xbox Kinect into this interaction. After that the obvious testing and tweaking.

-          As for visualisation, I plan to borrow code from Petra’s ‘Simple Digital Calligraphy’ and have a dynamic waveform that follows the movement of the participant. I haven’t started this and I’m guessing that a lot of experimentation is in order.

Assignment 2 - Research

These are a whole lot of works that I found inspirational; either visually or conceptually.

!!!PHENOMENAL!!!
http://www.creativeapplications.net/processing/multi-touch-sound-storm-processing-sound/ - Christian Bannister...this guy is my hero. With his 'Minority Report' style interface, he has taken interactive electronic music to a whole other level. Through this multi-touch display he is able to navigate, reverse, bit-slice and granulate sample tracks in real time. I would love to get my hands on one of these machines!

For Xbox Kinect
http://www.shiffman.net/2010/11/14/kinect-and-processing/ - Author of openKinect

Creative Applications Network
http://www.creativeapplications.net/ - Great resource for viewing new interactive works.

Time and Motion Study - John Tonkin
http://vimeo.com/5834154
http://vimeo.com/5834255 - 3D representation recorded video capture - Brilliant!

Great sound visualisations
http://www.youtube.com/watch?v=4rI3LGyD-3o - Painting with sound
http://www.youtube.com/watch?v=iF-2Fyie8z4 - 3D Energy waves
http://www.creativeapplications.net/processing/100-abandoned-artworks-processing/ - Beautiful aesthetic
http://www.creativeapplications.net/iphone/cloud-data-iphone-ipad/ - Data cloud!
http://www.creativeapplications.net/maxmsp/visual-music-collaborative-events-results/ - Great dynamic visualisations
http://www.creativeapplications.net/processing/nanokey-experiments-processing-sound/ - Relatively simple but very effective

Presentation
http://www.creativeapplications.net/processing/radius-music-processing/ - Nice use of light and shadow
http://www.creativeapplications.net/processing/rhifid-speakers-processing/ - Monitors, wires and space! Great presentation of a not-so-amazing concept ;)

Interactivity
http://www.creativeapplications.net/mac/gestural-music-sequencer-processing-sound/ - Amazing method of interacting using different coloured lights.

Animation and editing of video to sound
http://vimeo.com/3173439
http://vimeo.com/12771615

Assignment 2 - Ideas...

Idea 1 – Group Speak

A program that runs a line of text. A quote. The user reads the text hopefully at pace. It is then played back to the user as a composite with other people’s responses. Chopped up and changed around.

Idea 2 – Changed Perception of Reality

An augmented reality program similar to ‘Inception; program on iphone. Plays music and user wears a headset with a microphone near their mouth. Audio input is heavily layered with reverb while trippy visuals are found from the internet from around where they are. Purpose, changing the perception of the users immediate reality.

Idea 3 – Interactive Cloud of Sound

A simple interface with an audio waveform at the centre. Using a composite of soundscape’s, voices, and current sounds a ‘sound cloud’ is created.

When nobody is present, the program is relatively dormant with random outbursts of sound to trigger interest from passers-by. When the user approaches the screen they notice that they are able to affect this cloud. For example, their vertical height compared to the screen affects the high-pass frequency while the speed of their movements dictates the volume, etc, etc. Any sound that the user makes over a certain volume is then added to this cloud and is either immediately played back or saved for later.

Assignment 1 - 'Broken Audio Player'

See full sketch here!

EXPLAINATION 
* This sketch is a 'broken', visualisation based audio player. Moving the mouse up
* and down the y-axis controls a low-pass audio filter while pressing the transport
* controls reveals that the time will skip forwards and backwards on its own accord.
* Try and get to the end of the song!

Code borrowed from "Simple_digitalCalligraphy" by
* Petra Gemeinboeck and Rob Saunders (2009)
*
* Code borrowed from multiple 'minum' manual sketches at 
* http://code.compartmental.net/tools/minim/

More experimentation

This is just some simple experimentation with translation and mouse interaction. See the full sketch at http://www.openprocessing.org/visuals/?visualID=25878.

‘Transforming Mirrors’: Response to David Rokeby’s text

Today I was began reading ‘Transforming Mirrors: Subjectivity and Control in Interactive Media’ by David Rokeby, a recommended reading for my Intro to Interactivity class. Having only read halfway through the 16page paper I have already come across some good definitions and points to do with the origin, method and purpose of interactive works. Following are some quotes I found interesting:

“A technology is interactive to the degree that it reflects the consequences of our action or decision back to us. It follows that an interactive technology is a medium through which we communicate with ourselves…a mirror.” – Rokeby p1

“Interactive art is simply art that involves the participation of the viewer.” “All art can be called interactive in a deep sense if we consider viewing and interpreting a work of art as a kind or participation.” – Itsuo Sakane

“The spectator makes the picture” – Marcel Duchamp

“(The artist) anticipates the participant’s possible reactions and composes different relationships for each alternative.” – Myron Krueger

“The interactive system responds to the interactor, who in turn responds to that response. A feedback system is created in which the implications of an action are multiplied, much as we are reflected into infinity by the two facing mirrors in a barber shop.” – Rokeby p3-4

“The navigable structure can be thought of as an articulation of a space, either real, virtual or conceptual. The artist structures this space with a sort of architecture, and provides a method of navigation.” – Rokeby p4

First class!

After today's class I feel much better aquainted with the processing language. Without going into too much I thought I'd just put up a few sketches that I made in class.

Fig.1 70's design




Fig.2 - Death to the DJ. See the full one at OpenProcessing.org

Coding..

Hello! The purpose of this blog is as a journal for my Intro to Interactive Media class at COFA. It is a centralised location where I will post a variety of media that keep a regular record of my thoughts and artistic process throughout the semester. Lets get started!

Having completed the readings this week in 'Getting Started With Processing' I have realised two things. First of all coding is really quite confusing and secondly Processing gives you almost limitless potential with your project. I really feel that I need to get started with the concept for my "Transforming Mirror" project NOW if I am going to have any real chance of making something that the audience relates to.

I am very interested in sound and was really impressed by last year's 'glich' work where the MIDI track was played, reversed, skipped and was changed based on a combination of the user's interaction with the interface and random 'glichy' code functions. The aesthetic of the piece was very pleasing as well. I thoroughly enjoyed the lack of background, human identification and colourful MIDI notes.

Our lecturor Petra Hubanske, showed us a work by David Rokeby called 'Very Nervous System'. This really caught my attention because of the immersive interaction between the participant and the musical piece they are creating.

Something else that I am intrigued by is the XBOX Kinect. The fact that it has such accurate depth and positioning perception make it an amazing tool. Unfortunately, after talking to Petra I discovered that the current software only supports MAC OS which is unfortunate as I have a PC. Hopefully something will come up before too long and I'll be able to experiment with that as well!

Fig.1 is a screenshot of my first processing sketch where I was experimenting with drawing shapes, changing colours, stroke and layout. It really isn't that pretty but it is a start!

Fig. 1 - My first processing sketch.