Proxy-guided texture synthesis for rendering natural scenes

Landscapes and other natural scenes are easy to photograph but difficult to model and render. We present a proxy-guided pipeline which allows for simple 3D proxy geometry to be rendered with the rich visual detail found in a suitably pre-annotated example image. This greatly simplifies the geometric modeling and texture mapping of such scenes. Our method renders at near-interactive rates and is designed by carefully adapting guidance-based texture synthesis to our goals. A guidance-map synthesis step is used to obtain silhouettes and borders that have the same rich detail as the source photo, using a Chamfer distance metric as a principled way of dealing with discrete texture labels. We adapt an efficient parallel approach to the challenging guided synthesis step we require, providing a fast and scalable solution. We provide a solution for local temporal coherence, by introducing a reprojection algorithm, which reuses earlier synthesis results when feasible, as measured by a distortion metric. Our method allows for the consistent integration of standard CG elements with the texture-synthesized elements. We demonstrate near-interactive camera motion and landscape editing on a number of examples.