Image Sequence Restoration Using Gibbs Distributions

This thesis addresses a number of issues concerned with the restoration of one type of image sequence namely archived black and white motion pictures. These are often a valuable historical record but because of the physical nature of the film they can suffer from a variety of degradations which reduce their usefulness. The main visual defects are ‘dirt and sparkle’ due to dust and dirt becoming attached to the film or abrasion removing the emulsion and ‘line scratches’ due to the film running against foreign bodies in the camera or projector. For an image restoration algorithm to be successful it must be based on a mathematical model of the image. A number of models have been proposed and here we explore the use of a general class of model known as Markov Random Fields (MRFs) based on Gibbs distributions by analogy with models from statistical physics. The earliest such model was the Ising model and subsequently these models have been developed for use in other fields. The Gibbs distribution is a probability distribution over all possible images specified by local interactions between picture elements. The distribution is constructed such that of all possible images those with the desired characteristics are assigned higher probability. Because of the complexity inherent in these models a number of numerical techniques have been developed to use them for practical processing problems. The Metropolis-Hastings algorithm and the Gibbs sampler provide the basic tools constructing Markov Chains with the desired Gibbs distribution as the limiting distribution of the chain. These algorithms can also be used for other signal processing and inference problems. A number of deterministic approximations based on Mean Field theory are also used. The problem of detecting ‘dirt and sparkle’ is found to be equivalent to finding temporal discontinuities in the sequence corresponding to spurious motion estimates. A segmentation based approach working on motion estimates from a separate motion estimation algorithm is developed. To complete the restoration a different Markov Random Field model is used to interpolate the detected area the aim being to produce as seamless a restoration as possible. This approach treats the motion estimation and ‘dirt and sparkle’ detection as two separate processes. The flexibility of the Gibbs distributions which may be constructed allows a field to be specified which includes both the motion estimates and the segmentation in one distribution. The use of an MRF model on the motion estimates helps to produce a motion estimation algorithm which gives close approximations to the true motion and occlusion. This integrates the two processes into one estimation stage. The workings of this estimation algorithm are explored and applied to the problem of scratch detection. A slightly different approach is taken to the removal of ‘line scratches’. Their definite structure allows a strong model to be constructed. Because the number of line scratches present in any frame is unknown the standard Metropolis-Hastings and Gibbs sampler algorithms cannot be used. The Reversible Jump Markov chain Monte Carlo sampler is used to construct an algorithm for determining both the number and locations of the line scratches.