Motion Estimation and Compensation of Video Sequences using Affine Transforms

Motion estimation and compensation is of great importance for the compression of video sequences. In this dissertation a motion estimation/compensation approach based on a non-overlapping connected mesh of triangles is proposed. To manipulate the triangles within the connected mesh or ‘rubber sheet’ structure affin transforms are used which allow many different types of motion to be accurately modelled. Another advantage of this structure is that the non-overlapping triangles do not generate the typical artefacts associated with the current block based standards when operating at very low bitrates. The initial motion estimation/ compensation algorithms investigated implement a full search method which updates one vertex at a time matching sets of triangles between adjacent frames. Although the prediction performance is good the resulting computational load is high. This issue is addressed by deriving gradient-based algorithms which are found to be between one and two orders of magnitude faster than the equivalent full search methods whilst retaining an almost identical performance. The subjective quality of the reconstructed frames in those sequences containing signicant motion is improved by introducing a hierarchical approach into the affine motion estimation/compensation algorithm. To achieve better prediction results an additional processing stage is used. Two algorithms are implemented both of which relax the underlying connected mesh structure to allow triangles to overlap in certain areas of the image. The first method termed the ‘ripping’ algorithm allows sets of triangles to overlap along object boundaries (i.e. in those areas where occlusion or uncovering is thought to occur). The second method, termed the ‘advanced prediction’ algorithm, is based on a more general assumption that does not restrict the areas in which triangles are allowed to overlap. The single stage affine algorithms are found to be better in performance, both objectively and subjectively, than equivalent single stage translational algorithms such as H.261 and MPEG2. The two stage algorithm combining the affine hierarchical gradient-based stage with the translational advanced prediction stage is found to give as good as, or slightly better, subjective results and better objective results than the equivalent two-stage translational algorithms (i.e. H263).