Geometry Transfer for Stylizing Radiance Fields

CVPR 2024

1Seoul National University    2Meta Reality Labs

1-minute Intro

Abstract

Shape and geometric patterns are essential in defining stylistic identity. However, current 3D style transfer methods predominantly focus on transferring colors and textures, often overlooking geometric aspects. In this paper, we introduce Geometry Transfer, a novel method that leverages geometric deformation for 3D style transfer. This technique employs depth maps to extract a style guide, subsequently applied to stylize the geometry of radiance fields. Moreover, we propose new techniques that utilize geometric cues from the 3D scene, thereby enhancing aesthetic expressiveness and more accurately reflecting intended styles. Our extensive experiments show that Geometry Transfer enables a broader and more expressive range of stylizations, thereby significantly expanding the scope of 3D style transfer.

Qualitative Results

Given a reference 3D scene and a pair of style guides: an RGB image and a depth map ,
we coherently stylize both the scene’s appearance and shape to best express the given style.


Partial Stylization

Our method can be seamlessly incorporated into Panoptic Lifting, allowing us to freely render stylized object classes and instances according to our preferences during runtime.

Interpolation

The use of deformation fields, in conjunction with the original color grid, enables smooth interpolation between the original scene and the stylized one.

Comparisons

We compare our results with recent top-performing 3D style transfer methods on the fern scene from the LLFF dataset.

Content

Style