Notes on making a Doppler Mirror
Mark SubbaRao and Geza Gyuk, friends of mine at Adler Planetarium Astronomy Department, want to create a museum exhibit where you can dance in front of a Doppler Mirror; it displays you, but with parts of you tinted red or blue depending on whether that part is moving away or towards the 'mirror'.
One solution to make such a mirror - really a large video screen - is to place two video cameras so that they capture a stereo sequence of the museum visitor moving, and then use an algorithm that takes a pair of stereo images like this :
Stereo photographs of blueberries, iron-containing pebbles on Mars taken by the Rover Opportunity in 2004
and produces a 'depth' image like this
In a depth image, the value at each pixel says how far the corresponding point is from the cameras. Now we have a sequence of depth images, and their differences form a sequence of 'velocity' images. The velocity images can be used to tint the original image from the left or right camera red or blue. Finally, the sequence of velocity images is displayed on the large monitor that acts as the 'Mirror'.
For a NQR (not quite right) example, the picture below shows some rocks from Mars colored red-blue according to how far they are from the camera. This is NQR because the Doppler effect tints by velocity, not distance like the photo below does - we should tint with velocity images, not depth images.
The depth image above was computed with the open source C++ implementation of this Belief Propagation Algorithm by Pedro Felzenswalb (U Chicago) and Daniel P. Huttenlocher (Cornell). Their code is easy to use - and we didnt have to do any parameter tuning to get the above to work (we'll probably need to do some to get it to work faster though). When you compile it, an executable called 'stereo' is produced and then you can say from the command line
Where the image files are left/right/depthimage.pgm .
However, as we have already mentioned, this kind of algorithm is not the best way of doing it, as it does not use any temporal information to get depth.
Another solution is to use a single video camera and an Optical Flow algorithm. Such an algorithm takes a (non-stereo) video sequence as input and produces a velocity sequence. This is better for a museum environment since a single video camera is nice and cheap and low-maintenance - less calibration's needed!
While there's probably something called stereo optical flow, it may not be worth it for a Doppler Mirror.
Some links to follow up on:
One solution to make such a mirror - really a large video screen - is to place two video cameras so that they capture a stereo sequence of the museum visitor moving, and then use an algorithm that takes a pair of stereo images like this :
Stereo photographs of blueberries, iron-containing pebbles on Mars taken by the Rover Opportunity in 2004
and produces a 'depth' image like this
In a depth image, the value at each pixel says how far the corresponding point is from the cameras. Now we have a sequence of depth images, and their differences form a sequence of 'velocity' images. The velocity images can be used to tint the original image from the left or right camera red or blue. Finally, the sequence of velocity images is displayed on the large monitor that acts as the 'Mirror'.
For a NQR (not quite right) example, the picture below shows some rocks from Mars colored red-blue according to how far they are from the camera. This is NQR because the Doppler effect tints by velocity, not distance like the photo below does - we should tint with velocity images, not depth images.
The depth image above was computed with the open source C++ implementation of this Belief Propagation Algorithm by Pedro Felzenswalb (U Chicago) and Daniel P. Huttenlocher (Cornell). Their code is easy to use - and we didnt have to do any parameter tuning to get the above to work (we'll probably need to do some to get it to work faster though). When you compile it, an executable called 'stereo' is produced and then you can say from the command line
./stereo leftimage rightimage depthimage
Where the image files are left/right/depthimage.pgm .
However, as we have already mentioned, this kind of algorithm is not the best way of doing it, as it does not use any temporal information to get depth.
Another solution is to use a single video camera and an Optical Flow algorithm. Such an algorithm takes a (non-stereo) video sequence as input and produces a velocity sequence. This is better for a museum environment since a single video camera is nice and cheap and low-maintenance - less calibration's needed!
While there's probably something called stereo optical flow, it may not be worth it for a Doppler Mirror.
Some links to follow up on:
- Efficient Belief Propagation Algorithm by Felzenswalb and Huttenlocher (can be extended to optical flow)
- this Matlab code from Abhijit Ogale for stereo-to-depth and optical-flow. Provides compiled Matlab files (DLLs) for Windows.
- Sentience from Bob Mottram. Windows code made for Rodney the Robotic Plonker.
posted by Dinoj | 20:53
0 Comments:
Post a Comment
<< Home