Description

I was playing around with some AR markers the other day and came up with this idea. taking just a plain old vinyl record and attaching an AR marker to the label you can track the record in 3D space. The next question was, can you scratch the record?

So by figuring out the velocity of the records rotation and applying it to the payback of the audio you can scratch. There is some digital noise that needs to bee worked out, but sounds pretty good. Its still really hard to scratch, it takes some practice but is super fun. The next step is to figure out some nice triggers for different modes. I like the idea of not needing a turntable but the actual spinning of the record helps with the scratching and playback. I made a couple modes, one where the record is paused and you can just scratch through the song. The other looks for zero velocity for x time and then continues on with the song. If there is velocity you then are scratching and the audio is affected. I think that this project has some legs can’t wait to play more.

step 1:
You need to collect a bunch of negative images. Images that do not have the object that you are trying to track. You need to create a .dat file; a file that is just a list of all the images in a directory.

step 2:
There are a couple of techniques for step two. Opencv can generate positive samples from one image of the object you want to track, this is not the best technique but does work.
when you do this you get a bunch of images that look something like this.


The other approach is to have a series of positive images that are cropped. I think that this is the more accurate approach. I needed figure out how the openCv executable was building these samples. After figuring this out I build an app that loads a series of images that you can click all the objects in the image. You can also use a video camera to add more images and save out images. Here is a demo of the app working.

step 3:
Now you just need to train. Running the opencv haar training executable you will generate an .xml file to load in a haarTracking app. I am still learning all the settings for this step. The perfect tweak of the setting will give you the best result for what you want to track.

Here is what I have been able to get so far. The results are not to bad but I need to learn some more. Any help would be great.

We developed custom software using Openframeworks (openframeworks.cc), an open source development environment for C++. The software is displayed on a HD (1080 x 1920) vertical LCD screen, enclosed in a custom fabricated bamboo frame. The interaction is based on motion and presence. As people stand in front of the display a camera captures the them and the software analyzes the movement in each frame of the video. In realtime the users movement is translated in the a wind force. A physics system is the skeleton for all the elements is the environment. This force and physics system is what animates and brings the environment to life. As the user sways and moves the plants will move with you.

The interaction becomes a tango between you and the plants. To encourage growing we are isolating the upward motion and computing that into plant growth. The more someone moves up the more the plants will grow. The application has two modes day and night, at night the sky fades to a black gradient and stars twinkle in the background. When the sun rises the environment transitions back into a sunny day.

Installation Photos

Application Photos

Video

Credits

Todd Vanderlin
Ryan Habbyshaw
Memo Akten

ps: the gui is a new addon im working on download soon, or find it on the OF forum.

The app load a series of type images that I created. They are analyzed in realtime in order to fide the whole words and then each individual letter. I created many particle system that react to the each letter. I hope to work on the piece some more and do more of these events. Thanks Josh for getting this together, next stop Boston.


Q6M

0J4I

907

HG8

see all photos here

These creatures are attracted to people; they are curious. As you get closer they will jump out and have a taste. This piece is projected over a stairwell 10′x18′.

Video

So now that we have a corrected image to track I need to match the projection to the tabletop. We do a very simalar calibration, this does not feel correct but works. You drag the four points of a quad warping the openGL world to the table. This for the most part works but the distortion of the GL world is weird. There must be a better way to do this.

I have been looking into all the techniques done by other systems like TouchLib and Reactivision. Most are a four point or many point touch calibration system. I have not completely figured this out, this is where I ask for help. What is the best way to do this calibration and setup?

“We make you smile”
The piece allows you to be someone or something else for a moment. As the user walks by the piece finds their face and eyes replacing them with a new image. The reaction is priceless.

Video

Beta Video

Galaxy was an installation done for the Children’s Tumor Fund in NYC. The concept was to create an experience where people could change the atmosphere of the room by creating constellations.

A custom built multitouch table was placed in the middle of the room. People could draw shapes and and make connection with the clear blocks that symbolized children’s toys. Above their heads was a large canopy where stars and constellations were projected. As connections were made on the table constellations would appear above everyones head. Words that represented foundation would build if the right constellation was created. The piece allowed people to set the mood and transform the space in a new way.

Video