Simulation Methods for Linear and Nonlinear Time Series Models with Application to Distorted Audio Signals

This dissertation is concerned with the development of Markov chain Monte Carlo (MCMC) methods for the Bayesian restoration of degraded audio signals. First, the Bayesian approach to time series modelling is reviewed, then established MCMC methods are introduced. The first problem to be addressed is that of model order uncertainty. A reversible-jump sampler is proposed which can move between models of different order. It is shown that faster convergence can be achieved by exploiting the analytic structure of the time series model. This approach to model order uncertainty is applied to the problem of noise reduction using the simulation smoother. The effects of incorrect autoregressive (AR) model orders are demonstrated, and a mixed model order MCMC noise reduction scheme is developed. Nonlinear time series models are surveyed, and the advantages of linear-in- the-parameters models explained. A nonlinear AR (NAR) model, ...

Troughton, Paul Thomas — University of Cambridge

Accelerating Monte Carlo methods for Bayesian inference in dynamical models

Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society. Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change. In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions. The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions. To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution. These methods enjoy well-understood statistical properties but are often computational prohibitive to employ. The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo ...

Dahlin, Johan — Linköping University

Particle Filters and Markov Chains for Learning of Dynamical Systems

Sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) methods provide computational tools for systematic inference and learning in complex dynamical systems, such as nonlinear and non-Gaussian state-space models. This thesis builds upon several methodological advances within these classes of Monte Carlo methods. Particular emphasis is placed on the combination of SMC and MCMC in so called particle MCMC algorithms. These algorithms rely on SMC for generating samples from the often highly autocorrelated state-trajectory. A specific particle MCMC algorithm, referred to as particle Gibbs with ancestor sampling (PGAS), is suggested. By making use of backward sampling ideas, albeit implemented in a forward-only fashion, PGAS enjoys good mixing even when using seemingly few particles in the underlying SMC sampler. This results in a computationally competitive particle MCMC algorithm. As illustrated in this thesis, PGAS is a useful tool for both ...

Lindsten, Fredrik — Linköping University

Image Segmentation using Markov Random Field Models

The development of a fully unsupervised algorithm to achieve image segmentation is the central theme of this dissertation. Existing literature falls short of such a goal providing many algorithms capable of solving a subset of this highly challenging problem. Unsupervised segmentation is the process of identifying and locating the constituent regions of an observed image, while having no prior knowledge of the number of regions. The problem can be formulated in a Bayesian framework and through the use of an assumed model unsupervised segmentation can be posed as a problem of optimisation. This is the approach pursued throughout this dissertation. Throughout the literature, the commonly adopted model is an hierarchical image model whose underlying components are various forms of Markov Random Fields Gaussian. Markov Random Field models are used to model the textural content of the observed images regions, while ...

Barker, Simon A. — University of Cambridge

Image Sequence Restoration Using Gibbs Distributions

This thesis addresses a number of issues concerned with the restoration of one type of image sequence namely archived black and white motion pictures. These are often a valuable historical record but because of the physical nature of the film they can suffer from a variety of degradations which reduce their usefulness. The main visual defects are ‘dirt and sparkle’ due to dust and dirt becoming attached to the film or abrasion removing the emulsion and ‘line scratches’ due to the film running against foreign bodies in the camera or projector. For an image restoration algorithm to be successful it must be based on a mathematical model of the image. A number of models have been proposed and here we explore the use of a general class of model known as Markov Random Fields (MRFs) based on Gibbs distributions by ...

Morris, Robin David — University of Cambridge

Bayesian Fusion of Multi-band Images: A Powerful Tool for Super-resolution

Hyperspectral (HS) imaging, which consists of acquiring a same scene in several hundreds of contiguous spectral bands (a three dimensional data cube), has opened a new range of relevant applications, such as target detection [MS02], classification [C.-03] and spectral unmixing [BDPD+12]. However, while HS sensors provide abundant spectral information, their spatial resolution is generally more limited. Thus, fusing the HS image with other highly resolved images of the same scene, such as multispectral (MS) or panchromatic (PAN) images is an interesting problem. The problem of fusing a high spectral and low spatial resolution image with an auxiliary image of higher spatial but lower spectral resolution, also known as multi-resolution image fusion, has been explored for many years [AMV+11]. From an application point of view, this problem is also important as motivated by recent national programs, e.g., the Japanese next-generation space-borne ...

Wei, Qi — University of Toulouse

Improved State Estimation for Jump Markov Linear Systems

This thesis presents a comprehensive example framework on how current multiple model state estimation algorithms for jump Markov linear systems can be improved. The possible improvements are categorized as: -Design of multiple model state estimation algorithms using new criteria. -Improvements obtained using existing multiple model state estimation algorithms. In the first category, risk-sensitive estimation is proposed for jump Markov linear systems. Two types of cost functions namely, the instantaneous and cumulative cost functions related with risk-sensitive estimation are examined and for each one, the corresponding multiple model estate estimation algorithm is derived. For the cumulative cost function, the derivation involves the reference probability method where one defines and uses a new probability measure under which the involved processes has independence properties. The performance of the proposed risk-sensitive filters are illustrated and compared with conventional algorithms using simulations. The thesis addresses ...

Orguner, Umut — Middle East Technical University

Bayesian Approaches in Image Source Seperation

In this thesis, a general solution to the component separation problem in images is introduced. Unlike most existing works, the spatial dependencies of images are modelled in the separation process with the use of Markov random fields (MRFs). In the MRFs model, Cauchy density is used for the gradient images. We provide a general Bayesian framework for the estimation of the parameters of this model. Due to the intractability of the problem we resort to numerical solutions for the joint maximization of the a posteriori distribution of the sources, the mixing matrix and the noise variances. For numerical solution, four different methods are proposed. In first method, the difficulty of working analytically with general Gibbs distributions of MRF is overcome by using an approximate density. In this approach, the Gibbs distribution is modelled by the product of directional Gaussians. The ...

Kayabol, Koray — Istanbul University

Estimation of Nonlinear Dynamic Systems: Theory and Applications

This thesis deals with estimation of states and parameters in nonlinear and non-Gaussian dynamic systems. Sequential Monte Carlo methods are mainly used to this end. These methods rely on models of the underlying system, motivating some developments of the model concept. One of the main reasons for the interest in nonlinear estimation is that problems of this kind arise naturally in many important applications. Several applications of nonlinear estimation are studied. The models most commonly used for estimation are based on stochastic difference equations, referred to as state-space models. This thesis is mainly concerned with models of this kind. However, there will be a brief digression from this, in the treatment of the mathematically more intricate differential-algebraic equations. Here, the purpose is to write these equations in a form suitable for statistical signal processing. The nonlinear state estimation problem is ...

Schon, Thomas — Linkopings Universitet


In this thesis a number of improvements have been established for specific methods which utilize sequential Monte Carlo (SMC), aka. Particle filtering (PF) techniques. The first problem is the Bayesian multi-target tracking (MTT) problem for which we propose the use of non-parametric Bayesian models that are based on time varying extension of Dirichlet process (DP) models. The second problem studied in this thesis is an important application area for the proposed DP based MTT method; the tracking of vocal tract resonance frequencies of the speech signals. Lastly, we investigate SMC based parameter estimation problem of nonlinear non-Gaussian state space models in which we provide a performance improvement for the path density based methods by utilizing regularization techniques.

Ozkan, Emre — Middle East Technical University

Adaptive filtering algorithms for acoustic echo cancellation and acoustic feedback control in speech communication applications

Multimedia consumer electronics are nowadays everywhere from teleconferencing, hands-free communications, in-car communications to smart TV applications and more. We are living in a world of telecommunication where ideal scenarios for implementing these applications are hard to find. Instead, practical implementations typically bring many problems associated to each real-life scenario. This thesis mainly focuses on two of these problems, namely, acoustic echo and acoustic feedback. On the one hand, acoustic echo cancellation (AEC) is widely used in mobile and hands-free telephony where the existence of echoes degrades the intelligibility and listening comfort. On the other hand, acoustic feedback limits the maximum amplification that can be applied in, e.g., in-car communications or in conferencing systems, before howling due to instability, appears. Even though AEC and acoustic feedback cancellation (AFC) are functional in many applications, there are still open issues. This means that ...

Gil-Cacho, Jose Manuel — KU Leuven

Identification using Convexification and Recursion

System identification studies how to construct mathematical models for dynamical systems from the input and output data, which finds applications in many scenarios, such as predicting future output of the system or building model based controllers for regulating the output the system. Among many other methods, convex optimization is becoming an increasingly useful tool for solving system identification problems. The reason is that many identification problems can be formulated as, or transformed into convex optimization problems. This transformation is commonly referred to as the convexification technique. The first theme of the thesis is to understand the efficacy of the convexification idea by examining two specific examples. We first establish that a l1 norm based approach can indeed help in exploiting the sparsity information of the underlying parameter vector under certain persistent excitation assumptions. After that, we analyze how the nuclear ...

Dai, Liang — Uppsala University

Optimization Algorithms for Discrete Markov Random Fields, with Applications to Computer Vision

A large variety of important tasks in low­-level vision, image analysis and pat­tern recognition can be formulated as discrete labeling problems where one seeks to optimize some measure of the quality of the labeling. For example such is the case in optical flow estimation, stereo matching, image restoration to men­tion only a few of them. Discrete Markov Random Fields are ideal candidates for modeling these labeling problems and, for this reason, they are ubiquitous in computer vision. Therefore, an issue of paramount importance, that has at­tracted a significant amount of computer vision research over the past years, is how to optimize discrete Markov Random Fields efficiently and accurately. The main theme of this thesis is concerned exactly with this issue. Two novel MRF op­timization schemes are thus presented, both of which manage to extend current state-­of­-the­-art techniques in significant ways. ...

Komodakis, Nikos — University of Crete

Complex Baseband Modeling and Digital Predistortion for Wideband RF Power Amplifiers

Modern modulation methods as used in 3rd generation mobile communications (UMTS) generate strongly fluctuating transmission signal envelopes with high peak-to-average power ratios. These properties result in significant distortion due to the nonlinear behavior of the radio-frequency power amplifier (RF PA). We propose different nonlinear model structures for such amplifiers, based on memory polynomials and frequency-domain Volterra kernel expansion, where we can reduce the number of free parameters by 80% compared to traditional Volterra series approaches. Because these nonlinear models incorporate memory, we are able to model the nonlinear distortion of RF PAs with sufficient accuracy (e.g., −30 dB relative modeling error ), including the wideband case (bandwidth B = 20 MHz as needed for four-carrier WCDMA). Furthermore, we propose a method to construct RF PA models from frequency-dependent AM/AM and AM/PM conversions. For the compensation of the nonlinearities, we analyze ...

Singerl, Peter — Graz University of Technology

Signal Separation of Musical Instruments

This thesis presents techniques for the modelling of musical signals, with particular regard to monophonic and polyphonic pitch estimation. Musical signals are modelled as a set of notes, each comprising of a set of harmonically-related sinusoids. An hierarchical model is presented that is very general and applicable to any signal that can be decomposed as the sum of basis functions. Parameter estimation is posed within a Bayesian framework, allowing for the incorporation of prior information about model parameters. The resulting posterior distribution is of variable dimension and so reversible jump MCMC simulation techniques are employed for the parameter estimation task. The extension of the model to time-varying signals with high posterior correlations between model parameters is described. The parameters and hyperparameters of several frames of data are estimated jointly to achieve a more robust detection. A general model for the ...

Walmsley, Paul Joseph — University of Cambridge

The current layout is optimized for mobile phones. Page previews, thumbnails, and full abstracts will remain hidden until the browser window grows in width.

The current layout is optimized for tablet devices. Page previews and some thumbnails will remain hidden until the browser window grows in width.