Autonomous Systems and Robotics research centre
3D video-based reconstruction for Telepresence
Video-based reconstruction offers the potential of faithfully communicating both appearance and attention. Video conferencing can faithfully communicate what some looks like but not what he/she is looking at. Immersive collaborative virtual environment do the opposite. We are both building and studying the use of free viewpoint 3D video reconstruction from multiple 2D video streams. We have implemented both view dependent volumetric and view independent polyhedral approaches to reconstructing form.
We have implemented both reconstruction algorithms and multi-stream capture that allow real time frame rates for reconstructions from above 10HD colour cameras. We have built a unique test facility, the octave, which allows both immersive display and capture to support this research.
3D reconstruction using image manipulation and polyhedral based reconstruction
We create 3D models at real time frame rates using our parallelised implementation of an EPVL algorithm. Images are captured and backgrounds segments both in real time by our multi-video capture/manipulation system.
We have been studying the impact of camera placement on the ability of Vision Based Reconstruction to faithfully communicate appearance and attention.
The reconstructions displayed above were created from images captured within our octave display and capture simulation.
Silhouette cones used to reconstruct shape of what is being captured through multiple cameras.
Image displaying the impact of good camera placement on shape reconstruction
Impact of poor camera placement on shape reconstruction
- Duckworth, T & Roberts, D J 2011, Accelerated Polyhedral Visual Hulls using OpenCL, in: 'IEEE Virtual Reality', IEEE, Singapore, Singapore, pp.203-204.
- Duckworth, T & Roberts, D J 2011, Camera Image synchronization in Multiple Camera Real-time 3D Reconstruction of Moving Humans, in: 'Proceedings of 15th Int. Symp. On Distributed Simulation and Real Time Applications', IEEE, Salford, UK, pp.138-144.
- Moore, C & Duckworth, T & Roberts, D J 2011, Investigating the Suitability of a Software Capture Trigger in a 3D Reconstruction System for Telepresence, in: 'IEEE/ACM Proceedings of 15th Int. Symp. On Distributed Simulation and Real Time Applications', IEEE, Salford, UK, pp.134-137.
- Aspin, R & Roberts, D J 2011, A GPU based, projective multi-texturing approach to reconstructing the 3D human form for application in tele-presence, in: 'ACM Computer Supported Co-operative Working', ACM, New York, USA, pp.105-102
- Aspin, R & Roberts, D 2010, An Exploration of Non-tessellated 3D Space Carving for Real-Time 3D Reconstruction of a Person through a Simulated Process, in: 'Visual Media Production (CVMP), 2010 Conference on', IEEE, London, UK, pp.151-160. Conference details: Visual Media Production (CVMP), 2010 Conference on, London
- Moore, C & Duckworth, T & Aspin, R & Roberts, D 2010, Synchronization of Images from Multiple Cameras to Reconstruct a Moving Human, in: 'Distributed Simulation and Real Time Applications (DS-RT), 2010 IEEE/ACM 14th International Symposium on', IEEE/ACM, Fairfax, VA, USA, pp.53-60. Conference details: 2010 IEEE/ACM 14th International Symposium on Distributed Simulation and Real Time Applications (DS-RT), October 2010.