Novel View Synthesis using Diffusion Probabilistic Models

Bernard Spiegl’s MSc thesis funded by HIIT proposes a diffusion probabilistic framework for performing novel view synthesis.

Novel view synthesis is a class of computer vision problems where the main objective is synthesising new, previously unseen object or scene views based on one or multiple input views of a given object or a scene. The ability to synthesise new views is useful in various scenarios such as autonomous driving, object manipulation, trajectory predictions, augmenting already existing datasets, etc.

Bernard Spiegl’s MSc thesis introduces Palette View Synthesis, an end-to-end diffusion probabilistic generative modelling approach for performing novel view synthesis which aims to resolve the drawbacks of previous approaches by extending the model’s abilities to generalise across multiple classes, given only a single view and a target angle of the object as inputs, while simultaneously maintaining the quality of the generated samples. Unlike previous deterministic approaches, the stochasticity of diffusion process also allows for generation of several plausible novel views of the same object, while maintaining 3D consistency.

More information:

Alexander Ilin, Professor of Practice,

Department of Computer Science, Aalto University