Home Banner

Autonomous Systems and Robotics research centre

3D video-based reconstruction for Telepresence

Video-based reconstruction offers the potential of faithfully communicating both appearance and attention. Video conferencing can faithfully communicate what some looks like but not what he/she is looking at. Immersive collaborative virtual environment do the opposite. We are both building and studying the use of free viewpoint 3D video reconstruction from multiple 2D video streams. We have implemented both view dependent volumetric and view independent polyhedral approaches to reconstructing form.

Approach

We have implemented both reconstruction algorithms and multi-stream capture that allow real time frame rates for reconstructions from above 10HD colour cameras. We have built a unique test facility, the octave, which allows both immersive display and capture to support this research.

 3D reconstruction of David Roberts using image manipulation and polyhedral based reconstruction

3D reconstruction using image manipulation and polyhedral based reconstruction

Volumetric reconstruction

Volumetric reconstruction

We create 3D models at real time frame rates using our parallelised implementation of an EPVL algorithm. Images are captured and backgrounds segments both in real time by our multi-video capture/manipulation system.

We have been studying the impact of camera placement on the ability of Vision Based Reconstruction to faithfully communicate appearance and attention.

The reconstructions displayed above were created from images captured within our octave display and capture simulation.

Silhouette cone placement

Silhouette cones used to reconstruct shape of what is being captured through multiple cameras.

Impact of good camera placement

Image displaying the impact of good camera placement on shape reconstruction

Impact of poor camera placement 

Impact of poor camera placement on shape reconstruction

Publications

  • Duckworth, T & Roberts, D J 2011, Accelerated Polyhedral Visual Hulls using OpenCL, in: 'IEEE Virtual Reality', IEEE, Singapore, Singapore, pp.203-204.
  • Duckworth, T & Roberts, D J 2011, Camera Image synchronization in Multiple Camera Real-time 3D Reconstruction of Moving Humans, in: 'Proceedings of 15th Int. Symp. On Distributed Simulation and Real Time Applications', IEEE, Salford, UK, pp.138-144.
  • Moore, C & Duckworth, T & Roberts, D J 2011, Investigating the Suitability of a Software Capture Trigger in a 3D Reconstruction System for Telepresence, in: 'IEEE/ACM Proceedings of 15th Int. Symp. On Distributed Simulation and Real Time Applications', IEEE, Salford, UK, pp.134-137.
  • Aspin, R & Roberts, D J 2011, A GPU based, projective multi-texturing approach to reconstructing the 3D human form for application in tele-presence, in: 'ACM Computer Supported Co-operative Working', ACM, New York, USA, pp.105-102
  • Aspin, R & Roberts, D 2010, An Exploration of Non-tessellated 3D Space Carving for Real-Time 3D Reconstruction of a Person through a Simulated Process, in: 'Visual Media Production (CVMP), 2010 Conference on', IEEE, London, UK, pp.151-160. Conference details: Visual Media Production (CVMP), 2010 Conference on, London
  • Moore, C & Duckworth, T & Aspin, R & Roberts, D 2010, Synchronization of Images from Multiple Cameras to Reconstruct a Moving Human, in: 'Distributed Simulation and Real Time Applications (DS-RT), 2010 IEEE/ACM 14th International Symposium on', IEEE/ACM, Fairfax, VA, USA, pp.53-60. Conference details: 2010 IEEE/ACM 14th International Symposium on Distributed Simulation and Real Time Applications (DS-RT), October 2010.