Vision models and quality metrics for image processing applications

Optimizing the performance of digital imaging systems with respect to the capture, display, storage and transmission of visual information represents one of the biggest challenges in the field of image and video processing. Taking into account the way humans perceive visual information can be greatly beneficial for this task. To achieve this, it is necessary to understand and model the human visual system, which is also the principal goal of this thesis. Computational models for different aspects of the visual system are developed, which can be used in a wide variety of image and video processing applications. The proposed models and metrics are shown to be consistent with human perception. The focus of this work is visual quality assessment. A perceptual distortion metric (PDM) for the evaluation of video quality is presented. It is based on a model of the human visual system that takes into account color perception, the multi-channel architecture of temporal and spatial mechanisms, spatio-temporal contrast sensitivity, pattern masking as well as channel interactions, and accurately .ts data from psychophysical experiments. An in-depth evaluation of the performance of the proposed metric with respect to the prediction of perceived quality is carried out. Using extensive data from subjective experiments, the PDM is shown to perform well over a wide range of scenes and test conditions, and it compares favorably with competing metrics. Furthermore, the design choices for the different components of the PDM are analyzed with respect to their influence on prediction performance. Based on the results of this analysis, a number of extensions of the perceptual distortion metric are investigated. These include modifications of the PDM for the prediction of perceived blocking artifacts and for the support of object segmentation. Furthermore, attributes of image appeal are identified that contribute to perceived quality. They are integrated in the PDM in the form of sharpness and colorfulness ratings that are derived from the video sequence. Additional subjective experiments are carried out to establish a relationship between these ratings and perceived video quality. The combination of PDM predictions with sharpness and colorfulness ratings is shown to lead to improvements in prediction performance. Finally, a novel measure of local contrast is proposed. Contrast, i.e. the perception of stimuli in relation to their surround, is a fundamental aspect of vision and can facilitate numerous image processing and analysis tasks. The proposed definition is the first omnidirectional, phase-independent measure of local contrast that can be applied to natural images and agrees well with perceived contrast. Two specific applications of this contrast measure are presented. It is used as the basis for the above-mentioned sharpness rating and in the development of an image watermarking scheme based on a perceptual masking model. The evaluation of this scheme shows that it facilitates the insertion of watermarks with more energy. This leads to an increased robustness of the watermark while at the same time preserving the visual quality of the image.