In this dissertation a notional multi-sensor system acting in a maritime border control scenario for Homeland Security (HS) is analyzed, modelled, and simulated. The functions performed by the system are the detection, tracking, identification and classification of naval targets that enter a sea region, the evaluation of their threat level and the selection of a suitable reaction to them. The emulated system is composed of two platforms carrying multiple sensors: a land based platform, located on the coast, and an air platform, moving on an elliptic trajectory in front of the coast. The land based platform is equipped with a Vessel Traffic Service (VTS) radar, an infrared camera (IR) and a station belonging to an Automatic Identification System (AIS). The air platform carries an Airborne Early Warning Radar (AEWR) that can operate on a spotlight Synthetic Aperture Radar (SAR) mode, ...

Vision and infrared sensors are very common in surveillance and security applications, and there are numerous examples where a critical infrastructure, e.g. a harbor, an airport, or a military camp, is monitored by video surveillance systems. There is a need for automatic processing of sensor data and intelligent control of the sensor in order to obtain efficient and high performance solutions that can support a human operator. This thesis considers two subparts of the complex sensor fusion system; namely target tracking and sensor control.The multiple target tracking problem using particle filtering is studied. In particular, applications where road constrained targets are tracked with an airborne video or infrared camera are considered. By utilizing the information about the road network map it is possible to enhance the target tracking and prediction performance. A dynamic model suitable for on-road target tracking with ...

Localization, separation and tracking of acoustic sources are ancient challenges that lots of animals and human beings are doing intuitively and sometimes with an impressive accuracy. Artificial methods have been developed for various applications and conditions. The majority of those methods are centralized, meaning that all signals are processed together to produce the estimation results. The concept of distributed sensor networks is becoming more realistic as technology advances in the fields of nano-technology, micro electro-mechanic systems (MEMS) and communication. A distributed sensor network comprises scattered nodes which are autonomous, self-powered modules consisting of sensors, actuators and communication capabilities. A variety of layout and connectivity graphs are usually used. Distributed sensor networks have a broad range of applications, which can be categorized in ecology, military, environment monitoring, medical, security and surveillance. In this dissertation we develop algorithms for distributed sensor networks ...

In the modern world, we are increasingly surrounded by computation devices with communication links and one or more microphones. Such devices are, for example, smartphones, tablets, laptops or hearing aids. These devices can work together as nodes in an acoustic sensor network (ASN). Such networks are a growing platform that opens the possibility for many practical applications. ASN based speech enhancement, source localization, and event detection can be applied for teleconferencing, camera control, automation, or assisted living. For this kind of applications, the awareness of auditory objects and their spatial positioning are key properties. In order to provide these two kinds of information, novel methods have been developed in this thesis. Information on the type of auditory objects is provided by a novel real-time sound classification method. Information on the position of human speakers is provided by a novel localization ...

In the field of airborne earth observation there is special attention to compact, cost effective, high resolution imaging sensors. Such sensors are foreseen to play an important role in small-scale remote sensing applications, such as the monitoring of dikes, watercourses, or highways. Furthermore, such sensors are of military interest; reconnaissance tasks could be performed with small unmanned aerial vehicles (UAVs), reducing in this way the risk for one's own troops. In order to be operated from small, even unmanned, aircrafts, such systems must consume little power and be small enough to fulfill the usually strict payload requirements. Moreover, to be of interest for the civil market, cost effectiveness is mandatory. Frequency Modulated Continuous Wave (FMCW) radar systems are generally compact and relatively cheap to purchase and to exploit. They consume little power and, due to the fact that they are ...

This dissertation is dedicated to the exploration of various direct positioning algorithms for radio transmitters and passive target geolocation. Contrary to the traditional ``two-step'' approach, the ``direct positioning'' approach states that the radio transmitter's position can be extracted directly from the raw samples of the radio transmitter signals collected by the system sensors, without explicitly going through an estimation of position-related parameters such as time-delay, angular or amplitude information. In this work, the concept of direct positioning is applied to various models and consistently outperforms the traditional two-step position estimators, while tightly attaining the theoretical performance bounds. In the sequel, we explore 3 models for radio transmitters and passive target geolocation. The first model discussed in chapter 3, harnesses the transmit signal diversity of MIMO Radar systems to enhance passive-target position estimation via direct estimation algorithms. The algorithms are developed ...

Multisensor data fusion technology combines data and information from multiple sensors to achieve improved accuracies and better inference about the environment than could be achieved by the use of a single sensor alone. In this dissertation, we propose parametric and nonparametric multisensor data fusion algorithms with a broad range of applications. Image registration is a vital first step in fusing sensor data. Among the wide range of registration techniques that have been developed for various applications, mutual information based registration algorithms have been accepted as one of the most accurate and robust methods. Inspired by the mutual information based approaches, we propose to use the joint R´enyi entropy as the dissimilarity metric between images. Since the R´enyi entropy of an image can be estimated with the length of the minimum spanning tree over the corresponding graph, the proposed information-theoretic registration ...

Automated analysis of facial expressions has been an active area of study due to its potential applications not only for intelligent human-computer interfaces but also for human facial behavior research. To advance automatic expression analysis, this thesis proposes and empirically proves two hypotheses: (i) 3D face data is a better data modality than conventional 2D camera images, not only for being much less disturbed by illumination and head pose effects but also for capturing true facial surface information. (ii) It is possible to perform detailed face registration without resorting to any face modeling. This means that data-driven methods in automatic expression analysis can compensate for the confounding effects like pose and physiognomy differences, and can process facial features more effectively, without suffering the drawbacks of model-driven analysis. Our study is based upon Facial Action Coding System (FACS) as this paradigm ...

Super-resolution (SR) is a filtering technique that combines a sequence of under-sampled and degraded low-resolution images to produce an image at a higher resolution. The reconstruction takes advantage of the additional spatio-temporal data available in the sequence of images portraying the same scene. The fundamental problem addressed in super-resolution is a typical example of an inverse problem, wherein multiple low-resolution (LR)images are used to solve for the original high-resolution (HR) image. Super-resolution has already proved useful in many practical cases where multiple frames of the same scene can be obtained, including medical applications, satellite imaging and astronomical observatories. The application of super resolution filtering in consumer cameras and mobile devices shall be possible in the future, especially that the computational and memory resources in these devices are increasing all the time. For that goal, several research problems need to be ...

In today's society, we are flooded with massive volumes of data in the order of a billion gigabytes on a daily basis from pervasive sensors. It is becoming increasingly challenging to locally store and transport the acquired data to a central location for signal/data processing (i.e., for inference). To alleviate these problems, it is evident that there is an urgent need to significantly reduce the sensing cost (i.e., the number of expensive sensors) as well as the related memory and bandwidth requirements by developing unconventional sensing mechanisms to extract as much information as possible yet collecting fewer data. The first aim of this thesis is to develop theory and algorithms for data reduction. We develop a data reduction tool called sparse sensing, which consists of a deterministic and structured sensing function (guided by a sparse vector) that is optimally designed ...

This thesis introduces new learning algorithms for Radial Basis Function (RBF) networks. RBF networks is a feed-forward two-layer neural network used for functional approximation or pattern classification applications. The proposed training algorithms are based on robust statistics. Their theoretical performance has been assessed and compared with that of classical algorithms for training RBF networks. The applications of RBF networks described in this thesis consist of simultaneously modeling moving object segmentation and optical flow estimation in image sequences and 3-D image modeling and segmentation. A Bayesian classifier model is used for the representation of the image sequence and 3-D images. This employs an energy based description of the probability functions involved. The energy functions are represented by RBF networks whose inputs are various features drawn from the images and whose outputs are objects. The hidden units embed kernel functions. Each kernel ...

This dissertation details three approaches for direction-of-arrival (DOA) estimation or beamforming in array signal processing from the perspective of sparsity. In the first part of this dissertation, we consider sparse array beamformer design based on the alternating direction method of multipliers (ADMM); in the second part of this dissertation, the problem of joint DOA estimation and distorted sensor detection is investigated; and off-grid DOA estimation is studied in the last part of this dissertation. In the first part of this thesis, we devise a sparse array design algorithm for adaptive beamforming. Our strategy is based on finding a sparse beamformer weight to maximize the output signal-to-interference-plus-noise ratio (SINR). The proposed method utilizes ADMM, and admits closed-form solutions at each ADMM iteration. The algorithm convergence properties are analyzed by showing the monotonicity and boundedness of the augmented Lagrangian function. In addition, ...

This thesis addresses a number of problems all related to parameter estimation in sensor array processing. The unifying theme is that some of these parameters are known before the measurements are acquired. We thus study how to improve the estimation of the unknown parameters by incorporating the knowledge of the known parameters; exploiting this knowledge successfully has the potential to dramatically improve the accuracy of the estimates. For covariance matrix estimation, we exploit that the true covariance matrix is Kronecker and Toeplitz structured. We then devise a method to ascertain that the estimates possess this structure. Additionally, we can show that our proposed estimator has better performance than the state-of-art when the number of samples is low, and that it is also efficient in the sense that the estimates have Cramér-Rao lower Bound (CRB) equivalent variance. In the direction of ...

Mapping stationary objects and tracking moving targets are essential for many autonomous functions in vehicles. In order to compute the map and track estimates, sensor measurements from radar, laser and camera are used together with the standard proprioceptive sensors present in a car. By fusing information from different types of sensors, the accuracy and robustness of the estimates can be increased. Different types of maps are discussed and compared in the thesis. In particular, road maps make use of the fact that roads are highly structured, which allows relatively simple and powerful models to be employed. It is shown how the information of the lane markings, obtained by a front looking camera, can be fused with inertial measurement of the vehicle motion and radar measurements of vehicles ahead to compute a more accurate and robust road geometry estimate. Further, it ...

Wireless sensor networks has been one of the major research topics in recent years because of its great potential for a wide range of applications. In some application scenarios, sensor nodes intend to report the sensing data to a far-field destination, which cannot be realized by traditional transmission techniques. Due to the energy limitations and the hardware constraints of sensor nodes, distributed transmit beamforming is considered as an attractive candidate for long-range communications in such scenarios as it can reduce energy requirement of each sen-sor node and extend the communication range. However, unlike conventional beamforming, which is performed by a centralized antenna array, distributed beamforming is performed by a virtual antenna array composed of randomly located sensor nodes, each of which has an independent oscillator. Sensor nodes have to coordinate with each other and adjust their transmitting signals to collaboratively ...