Probabilistic graphical models are a powerful concept for representing complex relations between components in systems with uncertain behavior. Stemming originally from statistical mechanics, their applications stretch across various fields such as computer vision, speech recognition, social network analysis, and many more. Often they allow for a compact formulation of practical challenges in terms of fundamental computational problems, that are summarized under the term probabilistic inference. Unfortunately, these problems (e.g., the computation of marginal probabilities and the partition function) are computationally intractable so that we need to approximate the solution. In this thesis we consider two different categories of deterministic approximation methods: message passing algorithms and (variational) free energy approximations. Specifically, we focus on the most important representatives from either class, which are loopy belief propagation on the one hand, and the Bethe free energy approximation on the other hand. Although ...

In this thesis we investigate centralized and distributed variants of sparse Bayesian learning (SBL), an effective probabilistic regression method used in machine learning. Since inference in an SBL model is not tractable in closed form, approximations are needed. We focus on the variational Bayesian approximation, as opposed to others used in the literature, for three reasons: First, it is a flexible general framework for approximate Bayesian inference that estimates probability densities including point estimates as a special case. Second, it has guaranteed convergence properties. And third, it is a deterministic approximation concept that is even applicable for high dimensional problems where non-deterministic sampling methods may be prohibitive. We resolve some inconsistencies in the literature involved in other SBL approximation techniques with regard to a proper Bayesian treatment and the incorporation of a very desired property, namely scale invariance. More specifically, ...

Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society. Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change. In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions. The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions. To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution. These methods enjoy well-understood statistical properties but are often computational prohibitive to employ. The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo ...

The first part of this dissertation considers distributed learning problems over networked agents. The general objective of distributed adaptation and learning is the solution of global, stochastic optimization problems through localized interactions and without information about the statistical properties of the data. Regularization is a useful technique to encourage or enforce structural properties on the resulting solution, such as sparsity or constraints. A substantial number of regularizers are inherently non-smooth, while many cost functions are differentiable. We propose distributed and adaptive strategies that are able to minimize aggregate sums of objectives. In doing so, we exploit the structure of the individual objectives as sums of differentiable costs and non-differentiable regularizers. The resulting algorithms are adaptive in nature and able to continuously track drifts in the problem; their recursions, however, are subject to persistent perturbations arising from the stochastic nature of ...

Modeling dynamical systems is a fundamental task in various scientific and engineering domains, requiring accurate predictions, robustness to varying conditions, and interpretability of the underlying mechanisms. Traditional data-driven approaches often struggle with long-term prediction accuracy, generalization to out-of-distribution (OOD) scenarios, and providing insights into the system's behavior. This thesis explores the integration of supervised disentanglement into deep learning models as a means to address these challenges. We begin by advancing the state-of-the-art in modeling wave propagation governed by the Saint-Venant equations. Utilizing U-Net architectures and purposefully designed training strategies, we develop deep learning models that significantly improve prediction accuracy. Through OOD analysis, we highlight the limitations of standard deep learning models in capturing complex spatiotemporal dynamics, demonstrating how integrating domain knowledge through architectural design and training practices can enhance model performance. We further extend our supervised disentanglement approach to high-dimensional ...

This thesis investigates filter bank based multicarrier modulation using offset quadrature amplitude modulation (FBMC/OQAM), which is characterised by a critically sampled FBMC system that achieves full spectral efficiency in the sense of being free of redundancy. As a starting point, a performance comparison between FBMC/OQAM and oversampled (OS) FBMC systems is made in terms of per-subband fractionally spaced equalisation in order to compensate for the transmission distortions caused by dispersive channels. Simulation results show the reduced performance in equalising FBMC/OQAM compared to OS-FBMC, where the advantage for the latter stems from the use of guard bands. Alternatively, the inferior performance of FBMC/OQAM can be assigned to the inability of a per-subband equaliser to address the problem of potential intercarrier interference (ICI) in this system. The FBMC/OQAM system is analysed by representing the equivalent transmultiplexed channel including the filter banks as ...

In this thesis, we consider the problem of decentralized estimation under communication constraints in the context of Collaborative Signal and Information Processing. Motivated by sensor network applications, a high volume of data collected at distinct locations and possibly in diverse modalities together with the spatially distributed nature and the resource limitations of the underlying system are of concern. Designing processing schemes which match the constraints imposed by the system while providing a reasonable accuracy has been a major challenge in which we are particularly interested in the tradeoff between the estimation performance and the utilization of communications subject to energy and bandwidth constraints. One remarkable approach for decentralized inference in sensor networks is to exploit graphical models together with message passing algorithms. In this framework, after the so-called information graph of the problem is constructed, it is mapped onto the ...

In this thesis, smoothness of sampled real-world signals is exploited through the application of polynomial predictive filters. The principal reason for employing the polynomial signal model is principally twofold: firstly, assuming that the sampling rate is adequate, all real-world signals exhibit piecewise polynomial-like behavior, and secondly, polynomial-based signal processing is computationally efficient. By definition, polynomial predictive filters provide estimates of future values of polynomial-like signals. Thus, the potential applications of this research include a vast number of different delay sensitive operations on measurements like temperature, position, velocity, or power, especially in control engineering field. The polynomial-based predictive signal processing is a well-known technique, but polynomial-predictive filters have had severe drawbacks, which have hindered their application; their white noise attenuation is generally low, or they exhibit considerable passband gain peaks, rendering them unattractive for most applications. It has been possible to ...

Abstract Integrating AI into wireless network design and management is essential for creating self-sustaining 6G networks. A key challenge is the development of automated network procedures with minimal human intervention, leveraging real-time monitoring data for immediate feedback. These advancements promote data-driven decision-making but pose risks related to data availability, safety, and the black-box nature of learning algorithms. This cumulative thesis proposes and evaluates novel procedures and algorithms for data- driven radio planning and cellular network optimization, addressing practical challenges in applying learning-based methods on real-world deployments. It emphasizes the utility of monitoring data and the integration of model-based and model-free methods, ensuring the scalability and safety of adaptive network procedures across diverse environments. The first part of the thesis explores the application of deep learning to radio propagation modeling in live cellular networks. The first paper presents a novel network ...

Multimedia consumer electronics are nowadays everywhere from teleconferencing, hands-free communications, in-car communications to smart TV applications and more. We are living in a world of telecommunication where ideal scenarios for implementing these applications are hard to find. Instead, practical implementations typically bring many problems associated to each real-life scenario. This thesis mainly focuses on two of these problems, namely, acoustic echo and acoustic feedback. On the one hand, acoustic echo cancellation (AEC) is widely used in mobile and hands-free telephony where the existence of echoes degrades the intelligibility and listening comfort. On the other hand, acoustic feedback limits the maximum amplification that can be applied in, e.g., in-car communications or in conferencing systems, before howling due to instability, appears. Even though AEC and acoustic feedback cancellation (AFC) are functional in many applications, there are still open issues. This means that ...

In this dissertation we focus on statistical signal processing for Data Fusion, with a particular focus on wireless sensor networks. Six topics are studied: (i) Data Fusion for classification under model uncertainty; (ii) Decision Fusion over coherent MIMO channels; (iii) Performance analysis of Maximum Ratio Combining in MIMO decision fusion; (iv) Decision Fusion over non-coherent MIMO channels; (v) Decision Fusion for distributed classification of multiple targets; (vi) Data Fusion for inverse localization problems, with application to wideband passive sonar platform estimation. The first topic of this thesis addresses the problem of lack of knowledge of the prior distribution in classification problems that operate on small data sets that may make the application of Bayes' rule questionable. Uniform or arbitrary priors may provide classification answers that, even in simple examples, may end up contradicting our common sense about the problem. Entropic ...

We present the ‘gas of circles’ (GOC) model, which is a tool to describe a set of circles with an approximately fixed radius. The model is based on the recently introduced ‘higher-order active contour’(HOAC) framework. For certain ranges of the parameters, the model creates stable circles with an approximately fixed radius instead of networks. We show how to determine this set of parameters. The general ‘gas of circles’ model has many potential applications in varied domains, but it suffers from a drawback: the local minima corresponding to circles can trap the gradient descent algorithm, thus producing phantom circles even with no supporting data. We solve the problem of phantom circles by calculating, via a Taylor expansion of the energy, parameter values that make the circles into inflection points rather than minima. It is possible to create an alternative formulation of ...

This work aims at exploiting compressive sensing paradigms in order to reduce the cost of statistical learning tasks. We first provide a reminder of compressive sensing bases and describe some statistical analysis tasks using similar ideas. Then we describe a framework to perform parameter estimation on probabilistic mixture models in a case where training data is compressed to a fixed-size representation called a sketch. We formulate the estimation as a generalized inverse problem for which we propose a greedy algorithm. We experiment this framework and algorithm on an isotropic Gaussian mixture model. This proof of concept suggests the existence of theoretical recovery guarantees for sparse objects beyond the usual vector and matrix cases. We therefore study the generalization of linear inverse problems stability results on general signal models encompassing the standard cases and the sparse mixtures of probability distributions. We ...

In the last decade, cooperation among multiple terminals has been seen as one of the more promising strategies to improve transmission speed in wireless communications networks. Basically, the idea is to mimic an antenna array and apply distributed versions of well-known space-diversity techniques. In this context, the simplest cooperative scheme is the relay channel: all the terminals (relays) that overhear a point-to-point communication between a source and a destination may decide to aid the source by forwarding (relaying) its message. In a mobile system, it is common to assume that the relays do not have any information about the channel between them and the destination. Under this hypothesis, the best solution to exploit the diversity offered by multiple transmitting antennas is to use space-time coding (STC). However, classical STC's are designed for systems with a fixed and usually low number ...

This thesis considers the development of synchronization and signal processing techniques for digital communication receivers, which is greatly influenced by the digital revolution of electronic systems. Eventhough synchronization concepts are well studied and established in the literature, there is always a need for new algorithms depending on new system requirements and new trends in receiver architecture design. The new trend of using digital receivers where the sampling of the baseband signal is performed by a free running oscillator reduces the analog components by performing most of the functions digitally, which increases the flexibility, configurability, and integrability of the receiver. Also, this new design approach contributes greatly to the software radio (SWR) concept which is the natural progression of digital radio receivers towards multimode, multistandard terminals where the radio functionalities are defined by software. The first part of this research work ...