Video Friday: Markerless Tracking for Virtual Reality

The system uses a large number of videos taken from the amazing Carnegie Melon University camera dome. Using 480 cameras, arranged around the participant in a sort of dome, this system captures overlapping simultaneous videos from multiple perspectives. The many cameras are used to remove occlusions and generate a 3D representation of the person’s full body without the need for markers, equipment or any special garments. It’s an entirely unencumbered full body VR-chamber that can be used to dynamically control avatars, create 3D models, and more. Add a wireless head mounted display and you have a simple “holodeck” system. This is a pretty impressive result, and especially note the “confetti sequence” at the 2:25 mark. Wow. The technology goes beyond simply tracking the person and includes immediately also the tracking of carried items such as “props” which can be brought into the virtual world and even thrown objects. Hmm, we might have room for one over by the shed.

Screen Shot 2014-08-01 at 11.47.27 AMScreen Shot 2014-08-01 at 11.51.12 AM

Screen Shot 2014-08-01 at 11.48.06 AM

(above image from http://www.popphoto.com/news/2014/07/carnegie-mellon-packs-480-cameras-dome-to-perfectly-track-3d-motion)