In this thesis, a general solution to the component separation problem in images is introduced. Unlike most existing works, the spatial dependencies of images are modelled in the separation process with the use of Markov random fields (MRFs). In the MRFs model, Cauchy density is used for the gradient images. We provide a general Bayesian framework for the estimation of the parameters of this model. Due to the intractability of the problem we resort to numerical solutions for the joint maximization of the a posteriori distribution of the sources, the mixing matrix and the noise variances. For numerical solution, four different methods are proposed. In first method, the difficulty of working analytically with general Gibbs distributions of MRF is overcome by using an approximate density. In this approach, the Gibbs distribution is modelled by the product of directional Gaussians. The ...

In this dissertation we focus on statistical signal processing for Data Fusion, with a particular focus on wireless sensor networks. Six topics are studied: (i) Data Fusion for classification under model uncertainty; (ii) Decision Fusion over coherent MIMO channels; (iii) Performance analysis of Maximum Ratio Combining in MIMO decision fusion; (iv) Decision Fusion over non-coherent MIMO channels; (v) Decision Fusion for distributed classification of multiple targets; (vi) Data Fusion for inverse localization problems, with application to wideband passive sonar platform estimation. The first topic of this thesis addresses the problem of lack of knowledge of the prior distribution in classification problems that operate on small data sets that may make the application of Bayes' rule questionable. Uniform or arbitrary priors may provide classification answers that, even in simple examples, may end up contradicting our common sense about the problem. Entropic ...

Nowadays, many disciplines in science and engineering deal with problems for which a solution relies on knowledge about the characteristics of one or more given systems that can only be ascertained based on restricted observations. This requires the fitting of an adequately chosen model, such that it “best” conforms to a set of measured data. Depending on the context, this fitting procedure may resort to a huge amount of recorded data and abundant numerical power, or contrarily, to only a few streams of samples, which have to be processed on the fly at low computational cost. This thesis, exclusively focuses on the latter scenario. It specifically studies unexpected behaviour and reliability of the widely spread and computationally highly efficient class of gradient type algorithms. Additionally, special attention is paid to systems that combine several of them. Chapter 3 is dedicated ...

Array processing is an area of study devoted to processing the signals received from an antenna array and extracting information of interest. It has played an important role in widespread applications like radar, sonar, and wireless communications. Numerous adaptive array processing algorithms have been reported in the literature in the last several decades. These algorithms, in a general view, exhibit a trade-off between performance and required computational complexity. In this thesis, we focus on the development of array processing algorithms in the application of beamforming and direction of arrival (DOA) estimation. In the beamformer design, we employ the constrained minimum variance (CMV) and the constrained constant modulus (CCM) criteria to propose full-rank and reduced-rank adaptive algorithms. Specifically, for the full-rank algorithms, we present two low-complexity adaptive step size mechanisms with the CCM criterion for the step size adaptation of the ...

Hyperspectral (HS) imaging, which consists of acquiring a same scene in several hundreds of contiguous spectral bands (a three dimensional data cube), has opened a new range of relevant applications, such as target detection [MS02], classification [C.-03] and spectral unmixing [BDPD+12]. However, while HS sensors provide abundant spectral information, their spatial resolution is generally more limited. Thus, fusing the HS image with other highly resolved images of the same scene, such as multispectral (MS) or panchromatic (PAN) images is an interesting problem. The problem of fusing a high spectral and low spatial resolution image with an auxiliary image of higher spatial but lower spectral resolution, also known as multi-resolution image fusion, has been explored for many years [AMV+11]. From an application point of view, this problem is also important as motivated by recent national programs, e.g., the Japanese next-generation space-borne ...

Source coding concerns the representation of information in a source signal using as few bits as possible. In the case of lossy source coding, it is the encoding of a source signal using the fewest possible bits at a given distortion or, at the lowest possible distortion given a specified bit rate. Channel coding is usually applied in combination with source coding to ensure reliable transmission of the (source coded) information at the maximal rate across a channel given the properties of this channel. In this thesis, we consider the coding of auto-regressive (AR) sources which are sources that can be modeled as auto-regressive processes. The coding of AR sources lends itself to linear predictive coding. We address the problem of joint source/channel coding in the setting of linear predictive coding of AR sources. We consider channels in which individual ...

This doctorate thesis focuses on sparse regression, a statistical modeling tool for selecting valuable predictors in underdetermined linear models. By imposing different constraints on the structure of the variable vector in the regression problem, one obtains estimates which have sparse supports, i.e., where only a few of the elements in the response variable have non-zero values. The thesis collects six papers which, to a varying extent, deals with the applications, implementations, modifications, translations, and other analysis of such problems. Sparse regression is often used to approximate additive models with intricate, non-linear, non-smooth or otherwise problematic functions, by creating an underdetermined model consisting of candidate values for these functions, and linear response variables which selects among the candidates. Sparse regression is therefore a widely used tool in applications such as, e.g., image processing, audio processing, seismological and biomedical modeling, but is ...

Speech signals carry important information about a speaker such as age, gender, language, accent and emotional/psychological state. Automatic recognition of speaker characteristics has a wide range of commercial, medical and forensic applications such as interactive voice response systems, service customization, natural human-machine interaction, recognizing the type of pathology of speakers, and directing the forensic investigation process. This research aims to develop accurate methods and tools to identify different physical characteristics of the speakers. Due to the lack of required databases, among all characteristics of speakers, our experiments cover gender recognition, age estimation, language recognition and accent/dialect identification. However, similar approaches and techniques can be applied to identify other characteristics such as emotional/psychological state. For speaker characterization, we first convert variable-duration speech signals into fixed-dimensional vectors suitable for classification/regression algorithms. This is performed by fitting a probability density function to acoustic ...

The thesis deals with automatic interpretation of naturally spoken utterances for limited-domain applications. Specifically, the problem is examined by means of a dialogue system for an airport information application. In contrast to traditional two-stage systems, speech recognition and semantic processing are tightly coupled. This avoids interpretation errors due to early decisions. The presented one-stage decoding approach utilizes a uniform, stochastic knowledge representation based on weighted transition network hierarchies, which describe phonemes, words, word classes and semantic concepts. A robust semantic model, which is estimated by combination of data-driven and rule-based approaches, is part of this representation. The investigation of this hierarchical language model is the focus of this work. Furthermore, methods for modeling out-of-vocabulary words and for evaluating semantic trees are introduced.

In this thesis, we consider the problem of decentralized estimation under communication constraints in the context of Collaborative Signal and Information Processing. Motivated by sensor network applications, a high volume of data collected at distinct locations and possibly in diverse modalities together with the spatially distributed nature and the resource limitations of the underlying system are of concern. Designing processing schemes which match the constraints imposed by the system while providing a reasonable accuracy has been a major challenge in which we are particularly interested in the tradeoff between the estimation performance and the utilization of communications subject to energy and bandwidth constraints. One remarkable approach for decentralized inference in sensor networks is to exploit graphical models together with message passing algorithms. In this framework, after the so-called information graph of the problem is constructed, it is mapped onto the ...

Wireless networks and power distribution grids are experiencing increasing demands on their efficiency and reliability. Judicious methods for allocating scarce resources such as power and bandwidth are of paramount importance. As a result, nonlinear optimization and signal processing tools have been incorporated into the design of contemporary networks. This thesis develops schemes for efficient resource allocation (RA) in such dynamic networks, with an emphasis in stochasticity, which is accounted for in the problem formulation as well as in the algorithms and schemes to solve those problems. Stochastic optimization and decomposition techniques are investigated to develop low-complexity algorithms with specific applications in cross-layer design of wireless communications, cognitive radio (CR) networks and smart power distribution systems. The costs and constraints on the availability of network resources, together with diverse quality of service (QoS) requirements, render network design, management, and operation challenging ...

Sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) methods provide computational tools for systematic inference and learning in complex dynamical systems, such as nonlinear and non-Gaussian state-space models. This thesis builds upon several methodological advances within these classes of Monte Carlo methods. Particular emphasis is placed on the combination of SMC and MCMC in so called particle MCMC algorithms. These algorithms rely on SMC for generating samples from the often highly autocorrelated state-trajectory. A specific particle MCMC algorithm, referred to as particle Gibbs with ancestor sampling (PGAS), is suggested. By making use of backward sampling ideas, albeit implemented in a forward-only fashion, PGAS enjoys good mixing even when using seemingly few particles in the underlying SMC sampler. This results in a computationally competitive particle MCMC algorithm. As illustrated in this thesis, PGAS is a useful tool for both ...

Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society. Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change. In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions. The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions. To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution. These methods enjoy well-understood statistical properties but are often computational prohibitive to employ. The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo ...

In this study, the respiratory system are modelled by three linear and one non-linear lumped parameter respiratory model, the equations of the models are driven and the parameters are estimated by using statistical signal processing methods. Linear RIC, Viscoelastic and Mead models and proposed basic non-linear RC model are used to resemble the respiratory system of the patient with Chronic Obstructive Pulmonary Disease (COPD) under non-invasive ventilation. Statistical signal processing methods such as Minimum Variance Unbiased Estimation (MVUE), Maximum Likelihood Estimation (MLE), Kalman Filter (KF), Unscented Kalman Filter (UKF) and Extended Kalman Filter (EKF) are very powerful methods to estimate the parameters of the systems embedded in the unknown noise. In the first part of this thesis, artificial respiratory signals (airway flow and airway pressure) are used for the performance measurement criteria. Posterior Cramer Rao Lower Bound (PCRLB) is computed ...

System identification studies how to construct mathematical models for dynamical systems from the input and output data, which finds applications in many scenarios, such as predicting future output of the system or building model based controllers for regulating the output the system. Among many other methods, convex optimization is becoming an increasingly useful tool for solving system identification problems. The reason is that many identification problems can be formulated as, or transformed into convex optimization problems. This transformation is commonly referred to as the convexification technique. The first theme of the thesis is to understand the efficacy of the convexification idea by examining two specific examples. We first establish that a l1 norm based approach can indeed help in exploiting the sparsity information of the underlying parameter vector under certain persistent excitation assumptions. After that, we analyze how the nuclear ...