3D Photography promises to make 3D modeling as easy as point and shootWe all have cameras, and we know what photographs look like. They are two-dimensional representations of a real-world scene, taken from a specific point of view. But what if we had a camera that took three-dimensional pictures? What would a 3D photograph look like? How would it differ from 3D movies, View-Master slides, stereographic photographs, or other 3D images that create the illusion of depth?
 |
|
A true 3D photograph would contain information not only about an object's color and texture, but also its 3D physical shape, which could then be transformed into a 3D model. In fact, a goal of many researchers and manufacturers is to develop cameras that produce accurate 3D shapes, and ultimately 3D models, as their 'photographic' output. "Just as today's cameras capture appearance as planar images, cameras of the future will capture scene geometry and reflectance in the form of 3D photographs," says Steven Seitz, assistant professor of robotics and computer science at Carnegie Mellon University.
 |
|
The technology of 3D photography is moving closer to the day when this capability will be commonplace. Developments in optical engineering have resulted in 3D scanning devices that function much like traditional cameras in their ability to record the 3D shape of objects. And commercial software that can convert the 3D scan data into polygonal meshes, B-spline surfaces, or NURBS surfaces is more widely available.
 |
| A team at Stanford took 400 laser scans of Michelangelo's famous sculpture of David and stitched them together to create a 3D model containing some 2 billion polygons. (Image courtesy of the Digital Michelangelo Project, Stanford University.) |
There are many ways to construct 3D models of real-world objects, including using the powerful modeling tools available in a variety of CAD or 3D visualization software products. If a real-world object is complex, however, its geometrical surface is often too intricate and complicated to be accurately modeled with a software application.
One method for modeling such an object is to record the object surface coordinates with a digitizing position sensor. This procedure, used to digitize sculpted maquettes in the motion picture industry, can be done by a coordinate measuring machine (CMM) that mechanically establishes contact with the object and traces its 3D shape. Unfortunately, CMM contact methods require many surface points to ensure geometric accuracy and, therefore, are slow and tedious.
Alternately, non-contact digitization methods using 3D scanners are quicker, more accurate, less labor intensive, and more effective for modeling complex objects, particularly at extremely small or large size scales. As a result, a growing number of manufacturers have developed non-contact 3D scanning devices for acquiring 3D data in fields as diverse as precision manufacturing, historical reconstruction, art preservation, factory management, quality control, and entertainment.
 |
| A new technique called accordion fringe interferometry can quickly turn out high-precision models of complex objects. (Images courtesy of Dimensional Photonics.) |
The problem of determining the 3D shape of an existing object has been an ongoing research problem for years, particularly in the computer-vision community. " '3D photography' is an umbrella term for a wide variety of different techniques that fall into two classes-passive and active," says Seitz. "Passive methods work by interpreting the light already in the environment to infer geometry and material properties. Active methods work by casting light into the scene with lasers, projectors, or shadows, and studying the pattern of the light radiated back to the camera."
All of the currently available commercial 3D scanners use active techniques to illuminate and record the shapes of objects. The form of the laser light or projected white light used in active techniques can be a well-defined spot, a bright narrow line or stripe of light, a pattern, or full-area illumination. Active-illumination systems use this precisely formed light to illuminate an object's surface, and a digital still camera, CCD detector, or video camera to record the pattern created by the shape of that object in that light. After the detecting device acquires the pattern data, optical processing techniques recover the 3D geometry of the object. The geometry is then transformed into an accurate 3D model that can be exported to CAD programs or other 3D software applications.
Optical scientists and engineers have understood the principles of active illumination for years, but important industrial, commercial, and artistic uses for it have developed only recently. "The fundamental technologies for active sensing, such as laser stripe scanning, have not changed much in the last 10 years," says Marc Levoy, a computer graphics professor at Stanford University. "During this time, however, computers have become more powerful and memories have become larger, which allows researchers to scan larger objects. A few years ago we passed an invisible boundary between research-laboratory problems and real-world problems. It is now routine to scan sculptures and other objects, even large ones, and build non-trivial models for the design, manufacturing, and entertainment industries."
 |
| Scanning cameras produce a dense cloud of points that define the shape of objects such as this housing. Surfacing software converts the data into 3D models. |
The imaging techniques used by a 3D scanning device manufacturer usually depend on the market the manufacturer is serving, the size of the objects to be scanned, and the required accuracy of the measurement. Two important techniques are triangulation, which determines object distance by solving equations related to geometric position, and laser radar time of flight, which determines object distance by computing the time for a laser pulse to travel to the object and back to the detector.
Triangulation is essential for high-quality precision measurement, or 3D metrology, which is used for accurate micron-level quality control in manufacturing environments. The TriCam scanners from Perceptron are examples of triangulation scanning devices used in the automotive industry for defect detection and reverse engineering. The SOISIC scanner from Mensi is another triangulation device used in industrial and aerospace environments. Time-of-flight scanners are typically more effective at scanning objects and architectural spaces on a larger scale. The 100B scanner from MetricVision, the Cyrax scanner from Cyra, and the ZCAM scanner from 3DV Systems are examples of laser radar time-of-flight devices that are effective for digitizing buildings, bridges, factories, and room interiors.
Other systems are designed to scan objects within a particular size range. The Vantage from Cyberware and the Digibot II from Digibotics are desktop-size laser spot scanners that accurately scan smaller objects, about half a meter or less on each side. The FastScan from Polhemus and the ModelMaker from 3D Scanners are compact hand-held laser triangulation scanners designed to produce visually accurate 3D models quickly, particularly of hand-made maquettes in animation and video-game development where precise surface measurement is not a primary goal. Other 3D scanners are designed to serve specific target industries, such as the Voyager scanners from GSI Lumonics, which are widely used for quality control of circuit-board elements in microelectronic fabrication.
Making an active-illumination 3D scan is the first step in the creation of a 3D model. For most objects, multiple 3D scans are needed to digitize the model fully. Multiple scans, which depend on object complexity and can range anywhere from 3 to 100 in number, must be "stitched" together to form a single registered data set, just as multiple landscape photographs must be properly oriented to form a single panorama. When scan registration is complete, a surface must be constructed from the data points.
Scanning device manufacturers such as Cyra, Mensi, and InSpeck typically include registration and surface-reconstruction software with their devices. Several commercial software applications can also perform this registration and reconstruction function. CySlice from headus, Geomagic Studio from Raindrop Geomagic, Paraform from Paraform Inc., SurfaceStudio from Alias|Wavefront, and other packages each specialize in the software processing necessary to convert 3D point-cloud data into 3D models.
 |
| Given a relatively small number of data points, 3D model reconstruction software can recognize a variety of shapes and automatically render their surfaces. (Images courtesy of Mensi.) |
One major application of 3D photography that has emerged recently is industrial retrofitting. Often older machined parts or industrial plant layouts must be digitized and modeled in the process of revamping a factory system. In many cases, newer elements must be integrated into complicated existing structures for which CAD files, system drawings, or plot plans no longer exist. When older structures can be digitized and modeled with 3D scanning techniques, the integration of updated system elements into these older structures is swifter and less troublesome.
Unfortunately, 3D photography has not yet transformed the retrofitting industry, despite its enormous potential for cost cutting. That's because, in many scenarios, 3D photography of existing plant structures is still being combined with established labor-intensive methods of manual plant documentation to give precise "as-is" descriptions of a factory physical layout. "The technology of 3D photography is currently used on only a small percentage of revamp and retrofit projects to create a 3D model of an existing facility," contends Moh Hashemi, president of INOVx Solutions of Irvine, California. "In reality, the technology can be used as a complete replacement of manual physical data collection, at the same cost or lower, for revamp and retrofit projects, plant operation, and plant maintenance in almost every industry."
Often the problem with the adoption of 3D photography in industrial retrofitting is not with surface shape acquisition but with the data processing that results in a 3D model. "Creation of 3D models from 3D photography is manpower intensive," says Hashemi. "To alleviate this problem 3D photography should be used for creation of a broad physical database of 3D scans, which in conjunction with a powerful physical-database browser would be a good substitute for 3D modeling. In this approach the complex 3D modeling can be limited to the specific areas that require constructability analysis or clash detection."
What else limits the penetration of 3D scanning? "In a word, cost," says Edward Anderson, program manager with Air Products, Inc., of Pensacola, Florida, who uses 3D scanning devices in managing repairs and updates in process plant environments. "We have to make capturing the data so cheap that the return on investment can be calculated in months or weeks, not years. Plus, when the field data can be turned automatically into a 3D model via software, then the cost will really come down, and this work process will boom."
In another innovative application of 3D photography, a team headed by Levoy at Stanford recently started an ambitious project that expands the boundaries of 3D laser scanning and 3D model construction. This project, called the Digital Michelangelo Project, sent 30 researchers to Florence, Italy, to scan and digitize selected Michelangelo sculptures, including the unfinished St. Matthew and the famous figure of David in the Galleria dell'Accademia. The purpose of this project was to create a definitive set of highly detailed 3D models of these sculptures.
The team used 3D laser scanners, from Cyberware, Cyra, and 3D Scanners to create data sets of prodigious proportions. The large sculpture of David, for example, required 400 individual scans, resulting in a 3D model with 2 billion polygons. Since no commercial 3D application software can currently open a 3D model of that size, let alone render it, team members developed new and innovative tools, some of which are available at the Digital Michel-angelo Web site (graphics. stanford. edu/projects/mich), to pro-cess the multiple 3D scans into accurate 3D models.
There are many frontiers in 3D photography, and many problems to be solved. Pre-cision measurements are difficult to make, accurate scanning devices are expensive and bulky, and processing techniques are time consuming. No single scanning method dominates the field, and no single manufacturer has captured the marketplace. "The opportunities are vast," says Lyle Shirley, an optical physicist and president of Dimensional Photonics Inc., a Massachusetts-based 3D imaging start-up company. "But the applications for accurate 3D imaging systems are unlimited."
Consequently, exciting new developments in 3D photography are moving rapidly from research laboratory to market. In fact, Shirley is one of the inventors of a promising new 3D scanning technique called accordion fringe interferometry, or AFI, which illuminates an object with a series of laser-illuminated variable fringe patterns. Unlike other scanning devices, an AFI system can photograph objects of arbitrary size and complexity, with diverse material surfaces, and with high precision at any scale by projecting a special broad-beam light pattern. The system also has infinite depth of field, which avoids focus problems inherent in other devices, and it offers the possibility of high-resolution video-rate 3D imaging.
Other new 3D cameras are arriving in the marketplace that will appeal to users who need 3D visual accuracy rather than precision measurement. Recently, Minolta introduced the Minolta 3D 1500 digital camera with the MetaFlash lens and 3D image processing unit, which is designed to be a portable and easy-to-use device for creating simple 3D models such as images of commercial products that can be viewed on a company web site. The Venus3D camera from 3DMetrics and the 3Scan from Geometrix are similarly designed to offer convenient, low-cost methods for producing visually acceptable 3D models.
 |
| Accurate 3D models of priceless objects such as Michelangelo's unfinished St. Matthew can be used by historians, curators, and students to further scholarly research. (Image courtesy of the Digital Michelangelo Project, Stanford University.) |
What does all this development in 3D scanning and processing mean for the professional user of 3D photography? Will inexpensive commercial 3D cameras eventually become off-the-shelf commodities that are as easy to use as traditional cameras? This technology certainly has the potential to bring low-cost 3D desktop scanning to the professional market, a goal many researchers have dreamed about and worked toward for years. "A 3D imaging system coupled with a stereolithographic printer or rapid prototyping device would enable both 3D fax machines and 3D copy machines," declares Shirley. The concept of a high-precision single-box 3D scanning device is an exciting idea. It's the 3D camera of the future, and a picture we look forward to seeing.
Randall Warniers is managing editor of the Lincoln Laboratory Journal. He can be reached at warniers@ll.mit.edu.3DMetrics
Petaluma, CA
www.inharmonytech.com/venus3d
3D Scanners
London, England
www.3dscanners.com
3DV Systems
Yokneam, Israel
www.3dvsystems.com
Alias|Wavefront
Toronto, Ontario
www.aw.sgi.com
Cyberware
Monterey, CA
www.cyberware.com
Cyra Technologies
Oakland, CA
www.cyra.com
Digibotics
Austin, TX
www.digibotics.com
Dimensional Photonics
Boxborough, MA
www.dphotonics.com
Geometrix
San Jose, CA
www.geometrixinc.com
GSI Lumonics
Ann Arbor, MI
www.gsilumonics.com
headus
Osborne Park, WA
www.headus.com.au
InSpeck
Montreal, Quebec, Canada
www.inspeck.com
Mensi
Norcross, GA
www.mensi.com
MetricVision
Newington, VA
www.metricvision.com
Minolta
Ramsey, NJ
www.minolta3d.com
Paraform
Santa Clara, CA
www.paraform.com
Perceptron
Plymouth, MI
www.perceptron.com
Polhemus
Colchester, VT
www.polhemus.com
Raindrop Geomagic
Durham, NC
www.geomagic.com
Visual Interface
Pittsburgh, PA
www.visint.com