For those of you who don't know, I'm a graduate student in computer science. My latest (and ongoing) project is in stereo vision--that is, take two photographs of the same scene from slightly different angles, and try to reconstruct the geometry.

Despite being 25 years old I am a huge fan of Legos, so I chose when of my favorite Lego sets (Cafe Corner) the subject of the photographs that I'm testing my algorithm on. Here are the original two photos:

http://www.blake-foster.com/lego1.jpg
http://www.blake-foster.com/lego2.jpg

These two photos are passed to my algorithm, and from them, I get a 3d model of the scene. Here's the result view from a couple different angles:








Since it's a 3D model, I can view it from any angle, all computed from those two photos. It took about 8 minutes of computation time.

The software is written mostly in Matlab. The most computationally intensive part (computing matching points between the two images) uses a multithreaded C++ program. A few other parts use C++ as well.

Who isn't?

Very impressive project. If I provide you with multiple photos of my high-school math teacher, could you reconstruct him from squishy Legos?

I need a new punching bag

Thanks! Photos of a human being could be problematic, since the subject of the photos has to remain completely unchanged between shots.

Hi i also want to do one project, But i couldn't, before i start project what i should know, Please guide step by step

Here are your step by step instructions:

1. Go to a library.

That is a very impressive project. Great work!

Thanks! I'm working on the 3-view case now, which easily generalizes to multiple views.

How I wish I had that talent as well. Your project look realy nice!

How can you view the back from two front angles? It seems to me you can only view from the directions shot because you cannot extrapolate the 2 unseen sides accurately. Other than that, this project definitely looks fantastic!

Thanks! It all comes with practice.


Thanks! It's impossible to reconstruct the geometry in areas that the camera can't see. For now, I'm just detecting those regions and leaving holes in the mesh structure. Here's an example:

View #1:



View #2:



Screenshot of the 3D reconstruction (I've added texturing and mesh computation since the first pics ):



The big holes in the carpet behind the recipe book and the two boxes are present because the camera can't see those areas. I can still view the scene from any angle, but there will be holes in places that the camera can't see.

There is also a hole on top of the book, which is probably caused by the specular reflection.

The advantage of using more views (which I'm working on now) is that the algorithm is not only more robust, but there are also fewer areas that the camera can't see, and thus fewer holes in the mesh.

A scene point just needs to be visible in at least 2 views to get its 3D location.