Stereoscopic depth map estimation and coding techniques for multiview video systems

The dissertation deals with the problems of stereoscopic depth estimation and coding in multiview video systems, which are vital for development of the next generation three-dimensional television. The depth estimation algorithms known from literature, along with theoretical foundations are discussed. The problem of estimation of depth maps with high quality, expressed by means of accuracy, precision and temporal consistency, has been stated. Next, original solutions have been proposed. Author has proposed a novel, theoretically founded approach to depth estimation which employs Maximum A posteriori Probability (MAP) rule for modeling of the cost function used in optimization algorithms. The proposal has been presented along with a method for estimation of parameters of such model. In order to attain that, an analysis of the noise existing in multiview video and a study of inter-view correlation of corresponding samples of pictures have been done. Also, a novel technique for precision and accuracy enhancement of estimated depth maps is proposed. The technique employs an original Mid-Level Hypothesis algorithm which refines depth map in post-processing. Yet another, independent achievement of the dissertation is a novel technique for estimation of temporally consistent depth maps with use of noise removal from video prior to the depth estimation itself. In the dissertation, also, depth coding techniques are discussed. On a background of techniques known from the literature, the problem of depth representation suitable for coding, using legacy compression technology is stated. Author of the dissertation has proposed a novel method of representation of the depth, which employs non-linear transformation, which can be used in order to increase of compression performance in depth map coding. The proposed non-linear depth representation has been accepted by international group of experts (MPEG) and adopted to new 3D extensions of ISO/IEC 14496-10 and ITU Rec. H.264 international video coding standards, describing new generation of 3D video coding technologies, known under names of ?MVC+D? and ?AVC-3D?. All of the proposed algorithms have been implemented and their performance has been verified experimentally. The obtained results have been presented in the dissertation. The following theses have been formulated and proven: T1) Depth estimation can be improved by usage of modeling of the cost function based on maximization of a posteriori probability. T2) Precision and accuracy of estimated depth maps can be improved in post-processing with iterative insertion of intermediate values, controlled using view synthesis. T3) Temporal consistency of estimated depths can be improved using noise removal from input multiview video. T4) Non-linear representation of depth can be employed in order to improve compression efficiency of depth maps in 3D video systems. Additional achievement not related directly to the theses yet presented in the dissertation is author?s contribution to production of multiview video sequences that are currently broadly used for test purposes by international research teams, also in research done in the context of standardization in MPEG and JCT-3V expert groups. In the dissertation, also shown is author?s participation in the development of a 3D video codec, prepared at Chair of Multimedia Telecommunication and Microelectronics of Poznan University of Technology. The codec has been submitted as a proposal for ?Call for Proposals for 3D Video Coding Technology? issued by ISO/IEC MPEG group. The excellent results achieved by the proposed codec are shown on the background of exemplary proposals resulting from works of competitive research centers in the world.