Point Cloud Streaming for 3D Avatar Communication
Masaharu Kajitani, Shinichiro Takahashi, Masahiro Oku · InTech eBooks · 2008
IntroductionReal-time 3D imaging has gained special attention in many fields such as computer vision, tele-imaging, and virtual reality.Especially real-time 3D modeling is one of emerging topics.During the last years, many systems that reconstruct 3D images using image sequences obtained from several viewpoints have been proposed.In the work (Saito et al., 1999), the 3D image sequences are acquired by multiple images of about 50 cameras.They accomplish high quality 3D images.In the method, a modeling step is done offline.In (Wu & Matsuyama, 2003), (Ueda et al., 2004), (Würmlin et al., 2002), and(Würmlin et al., 2004), the 3D modeling is employed in real-time using parallel processing with PC clusters.All of them use Silhouette Volume Intersection (SVI) method to approximate a shape by a visual hull.In order to make the surface finer and map textures on it, some extra tasks are required, which are in principle time-consuming.On the other hand, some methods have been established to generate range images in realtime, for example (Kanade et al., 1996).Some of them have already been commercialized (Pointgrey).These methods achieve range images at 5-30 fps by the stereo matching algorithm.In this paper, we propose a client-server system in which the server reconstructs, encodes, and transmits the 3D images to client in real-time.We do not apply the polygonal modeling but transmit a point cloud of multiple range images and render it directly, due to the following two reasons: 1.To reconstruct mesh model, a triangulation is required.Moreover, one needs to extract textures from colour images, and map them onto the meshes, which are complicated and time-consuming.The computational cost of the processes depends on complexity of the object shapes.The point rendering does not need these.2. In general, the polygonal meshes have connectivity information as well as the 3Dcoordinates of the vertices.The connectivity has to be encoded without any loss, which consumes many bits.Moreover, only one bit error on the connectivity causes catastrophic damage on its quality.This is not suitable for transmission over wireless network or UDP.The point cloud can easily be divided and packetized to individual blocks.The client receives and decodes a bit-stream, and then renders the point cloud quickly by using a strip-based rendering.As a rendering method for the point cloud, Point-Sampled-Geometry (Rusinkiewicz & Levoy, 2000) is well known, which is used to represent high Open Access Database www.i-techonline.