Xuqian Ren "Building Digital Twins of Real Indoor Spaces from Consumer Devices"
This talk can be viewed via zoom. (Note: this talk will be recorded)
Abstract:
Constructing digital twins for AR/VR, robotics, and embodied AI requires accurate geometry, physically consistent appearance, and representations of surface materials. Recovering these properties from consumer video remains challenging due to image noise, blur, and incomplete scene coverage.
This talk presents my doctoral research addressing these challenges through four complementary directions. First, I introduce a real-world benchmark captured using smartphones, an RGB-D camera, and a laser scanner, demonstrating that rendering quality does not necessarily reflect geometric accuracy. Second, I present methods that selectively incorporate reliable depth and normal priors into Gaussian Splatting to improve surface and mesh reconstruction. Third, I describe an approach for integrating relightable human avatars into new scenes with consistent illumination and shadows. Fourth, I present evidence that video foundation models encode material information and discuss how these representations can support more scalable material estimation. Together, these studies examine the geometric, photometric, and material information needed to develop digital twins from consumer observations.
The talk concludes with future research directions towards interactive, physically faithful 3D worlds. These include decomposing scenes into editable objects, estimating physical properties from video, and developing autonomous AI agents for scene reconstruction, verification, and repair.
Bioļ¼
Xuqian Ren is a doctoral researcher at Tampere University, Finland, supervised by Prof. Esa Rahtu and Prof. Juho Kannala. Her research focuses on building digital twins of real indoor environments using data captured with consumer devices. Her work spans 3D reconstruction and generation, novel view synthesis, physically based rendering, and, more recently, AI agents. Her research has been published at WACV and 3DV.