Description

This is a project done in collaboration with YesYesNo. The goal was to turn the historic Ferry Building in downtown Auckland New Zealand into a giant interactive playground.

We used 3 different types of interaction – body interaction on the two stages, hand interaction above a light table, and phone interaction with the tracking of waving phones. There were 6 scenes, cycled every hour for the public. The whole experience lasted 6 nights.

Monsters

The monster scene was a full body interaction – as people danced in front of light boxes their contours would magically transform into a playful monster. As groups of people came together they created giant monsters. see more monsters here

Ferns

This scene is another full body interaction – as people made growing gestures a fern would spawn from their contour. Throughout the scene colorful butterflies would flock toward peoples bellies.

Paint

This scene allowed people to finger paint on the entire building.

Constellations

This scene generated large constellations from the motion of people’s cellphones. The more movement people made caused stars to spill from the building.

Photos

Credits

Interaction Design and Software: YesYesNo Joel Gethin Lewis, Zach Lieberman, Pete Hellicar, Kyle McDonald, Todd Vanderlin w/ Daito Manabe sound design

Projection / Staging: Electric Canvas, Inside Out Productions. Production: The Church, Simon Velvin. Production & Art Direction: Hellicar and Lewis

Thanks to NZ Telecom and the Auckland City Council for supporting this, Peter Milne and all the Team, Simon Velvin and all at the Church, Mike Mizrahi, Marie Adams and all at Inside Out Productions, and Takayuki Ito.

Description

The interactive aquarium is installed in store front windows in 6 cities across the US. Using computer vision the seascape will react to the motion of a user, seaweed will sway and fish will scatter. Users can then dial in with any mobile device and create a fish using their voice. As they connect in realtime the sounds they make are analyzed and create a dynamically generated fish. You can then enter that fish into the aquarium and use your key pad to eat various pieces of fish food that will transform and morph your fish into creatures. The application will remember the state of your fish so that when you call back later you can evolve your fish even further.

Technical

The installation was setup in two situations an HD rear projection and a LCD wall. The vision technology was able to track motion and trigger animations. The phone technology was built using megaphone a realtime multiuser mobile phone server.

Credits

Ryan Habbyshaw
Illustrations by Emily Gobeille
Arnold R&D

more photos

update: It even works on cats via: link

step 1:
You need to collect a bunch of negative images. Images that do not have the object that you are trying to track. You need to create a .dat file; a file that is just a list of all the images in a directory.

step 2:
There are a couple of techniques for step two. Opencv can generate positive samples from one image of the object you want to track, this is not the best technique but does work.
when you do this you get a bunch of images that look something like this.


The other approach is to have a series of positive images that are cropped. I think that this is the more accurate approach. I needed figure out how the openCv executable was building these samples. After figuring this out I build an app that loads a series of images that you can click all the objects in the image. You can also use a video camera to add more images and save out images. Here is a demo of the app working.

step 3:
Now you just need to train. Running the opencv haar training executable you will generate an .xml file to load in a haarTracking app. I am still learning all the settings for this step. The perfect tweak of the setting will give you the best result for what you want to track.

Here is what I have been able to get so far. The results are not to bad but I need to learn some more. Any help would be great.

These creatures are attracted to people; they are curious. As you get closer they will jump out and have a taste. This piece is projected over a stairwell 10′x18′.

Video

Screen Shots from the project

Lines
These are screen shots from “lines” an application that I built. There are two version one that builds based off of the computer. The other that is built based on your movement. I will Post downloads once I clean up the patches.