good day's work

Since I got control of myself again, I've been doing quite a bit of work, thank goodness. I've been writing out all this analysis by syntheis stuff, and today I did something I didnt expect to do this quickly, namely test the simplest case. It's proving tough going too, so it's a good thing I'm doing it now.

In other news, it seems there's a stereo video camera available on campus - perhaps we can play with it. Turns out what I should have been googling for before was "stereo vision camera" instead of "stereo video camera".

Applied for a free mediawiki called prettyscience.

The Superbowl was yesterday. In unrelated and far more relevant news, my team lost. Chelsea 2 - Liverpool 0. Robbie didnt play - some problem with fitness was the claim.

Was forced to return the water cooler/heater last week. Now walking two blocks to fill my mug with tea at night.

long time no blog

I've not really blogged about me the past two weeks, for good reasons - it's hard to blog about very interesting things, and very boring things, in that chronological order. For the record, shi(f)t happened.

I'm back now, and that feels good. Decided to start implementing a fast linear svm today - it's one of those rare situations that the author of the paper actually gives all details, even the minor ones, required to code something. Meanwhile, designing a general c++ library is hard.

I'm hoping to be able to get to Australia for EMNLP this year. I'd given up on going, as ACL's impossible to get into, but learnt yesterday that EMNLP is going to be there as well. I'll probably have to pay for most of my air fare though. At least accommodation will be free, though!

Looking forward to On-The-Media this week, since Xeni Jardin's cohosting it. She blogs on Boingboing.

Will have to get back to the thesis soon. Terrified that whatever I try won't work. And keep wanting to do other things, like the linear svm mentioned above. But that's important...

Did a lot of writing this week, including wikipedia entries on the Adler Astro wiki (I'll link to that when Mark sorts the permissions issues out) and my other hobby.

Ran three miles today, legs feel okay - for now. Ordered a new pair of running shoes too, so I can try running outside again. Found some New Balances cheap online ($35 including shipping) - they've worked for me before - hope I got the right size. I feel I can only run with my old tattered takkies on treadmills, and I'm tired of having to wait till 11 till I can get on one at the Ratner without having to wait.

It feels very strange to be in control of myself again. Is this what being normal feels like? How unusual. It's almost pleasant.

Notes on making a Doppler Mirror

Mark SubbaRao and Geza Gyuk, friends of mine at Adler Planetarium Astronomy Department, want to create a museum exhibit where you can dance in front of a Doppler Mirror; it displays you, but with parts of you tinted red or blue depending on whether that part is moving away or towards the 'mirror'.

One solution to make such a mirror - really a large video screen - is to place two video cameras so that they capture a stereo sequence of the museum visitor moving, and then use an algorithm that takes a pair of stereo images like this :


Stereo photographs of blueberries, iron-containing pebbles on Mars taken by the Rover Opportunity in 2004


and produces a 'depth' image like this





In a depth image, the value at each pixel says how far the corresponding point is from the cameras. Now we have a sequence of depth images, and their differences form a sequence of 'velocity' images. The velocity images can be used to tint the original image from the left or right camera red or blue. Finally, the sequence of velocity images is displayed on the large monitor that acts as the 'Mirror'.

For a NQR (not quite right) example, the picture below shows some rocks from Mars colored red-blue according to how far they are from the camera. This is NQR because the Doppler effect tints by velocity, not distance like the photo below does - we should tint with velocity images, not depth images.





The depth image above was computed with the open source C++ implementation of this Belief Propagation Algorithm by Pedro Felzenswalb (U Chicago) and Daniel P. Huttenlocher (Cornell). Their code is easy to use - and we didnt have to do any parameter tuning to get the above to work (we'll probably need to do some to get it to work faster though). When you compile it, an executable called 'stereo' is produced and then you can say from the command line

          ./stereo leftimage rightimage depthimage

Where the image files are left/right/depthimage.pgm .

However, as we have already mentioned, this kind of algorithm is not the best way of doing it, as it does not use any temporal information to get depth.

Another solution is to use a single video camera and an Optical Flow algorithm. Such an algorithm takes a (non-stereo) video sequence as input and produces a velocity sequence. This is better for a museum environment since a single video camera is nice and cheap and low-maintenance - less calibration's needed!

While there's probably something called stereo optical flow, it may not be worth it for a Doppler Mirror.

Some links to follow up on:

• Efficient Belief Propagation Algorithm by Felzenswalb and Huttenlocher (can be extended to optical flow)
  
• this Matlab code from Abhijit Ogale for stereo-to-depth and optical-flow. Provides compiled Matlab files (DLLs) for Windows.
  
• Sentience from Bob Mottram. Windows code made for Rodney the Robotic Plonker.