Make3D: learning 3D scene structure from a single still image.

Full Abstract

We consider the problem of estimating detailed 3D structure from a single still image of an unstructured environment. Our goal is to create 3D models that are both quantitatively accurate as well as visually pleasing. For each small homogeneous patch in the image, we use a Markov Random Field (MRF) to infer a set of "plane parameters" that capture both the 3D location and 3D orientation of the patch. The MRF, trained via supervised learning, models both image depth cues as well as the relationships between different parts of the image. Other than assuming that the environment is made up of a number of small planes, our model makes no explicit assumptions about the structure of the scene; this enables the algorithm to capture much more detailed 3D structure than does prior art and also give a much richer experience in the 3D flythroughs created using image-based rendering, even for scenes with significant nonvertical structure. Using this approach, we have created qualitatively correct 3D models for 64.9 percent of 588 images downloaded from the Internet. We have also extended our model to produce large-scale 3D models from a few images.

Author information

Author/s: Saxena, Ashutosh (A); Sun, Min (M); Ng, Andrew Y (AY);

Affiliation: Computer Science Department,Stanford University, Gates Building 1A, Computer Science, 353 SerraMall, Stanford, CA 94305-9010. {asaxena, ang}(-atsign-)cs.stanford.edu

Journal and publication information

Publication Type: Journal Article; Research Support, U.S. Gov't, Non-P.H.S.

Journal: IEEE transactions on pattern analysis and machine intelligence (IEEE Trans Pattern Anal Mach Intell), published in United States. (Language: eng)

Reference: 2009-May; vol 31 (issue 5) : pp 824-40

Dates: Created 2009/03/20; Completed 2009/06/10;

PMID: 19299858, status: MEDLINE (last retrieval date: 6/10/2009, IMS Date: 10 Jun 2009 00:00:00)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article
(including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

Algorithms
Artificial Intelligence
Image Enhancement - methods
Image Interpretation, Computer-Assisted - methods
Imaging, Three-Dimensional - methods

Information Storage and Retrieval - methods
Pattern Recognition, Automated - methods
Photography - methods
Reproducibility of Results
Sensitivity and Specificity

This article has not been indexed for related articles as yet, however you can still use the live related article search links below.

See 100+ related articles.

See a large map of 100+ related articles.