Second-Order Multidimensional Independent Component Analysis: Theory and Methods

Independent component analysis (ICA) and blind source separation (BSS) deal with extracting a number of mutually independent elements from a set of observed linear mixtures. Motivated by various applications, this work considers a more general and more flexible model: the sources can be partitioned into groups exhibiting dependence within a given group but independence between two different groups. We argue that this is tantamount to considering multidimensional components, as opposed to the standard ICA case which is restricted to one-dimensional components. In this work, we focus on second-order methods to separate statistically-independent multidimensional components from their linear instantaneous mixture. The purpose of this work is to provide theoretical answers to questions which so far have been discussed mainly in the empirical domain. Namely, we provide a closed-form expression for the figure of merit, the mean square error (MSE for multidimensional component separation, in two prominent scenarios: one is the optimal separation procedure, and the other is a two-steps procedure, which at the first step attributes the data a one-dimensional model, and at the second step clusters the data into multidimensional components. We prove that the latter is sub-optimal. Using the closed-form expressions, we can calculate the expected gain from using the correct multidimensional model over its one-dimensional counterpart. We present new results on the identifiability of the model, including a rigorous proof. We derive two novel joint block diagonalization (JBD) algorithms which achieve the minimal mean square error (MMSE), when the model assumptions hold. The identifiability results of the model are also the condition for the uniqueness and existence of a solution to JBD of a set of positive definite symmetric matrices. As for the methods used in this work, it is shown that all the required derivations can be performed and presented in terms of well-defined quantities, which avoid the well-known scale ambiguity, prevalent in ICA and BSS representations. We demonstrate our methods and algorithms on an astrophysical application. Namely, extraction of the cosmic microwave background radiation (CMB) from its observations. We adapt our analysis to the spectral domain, in order to apply our methods on spectral density matrices. In this application, we extend our analysis to the overdetermined case, and deal with a situation where the component dimensions are not known a-priori. We validate our results using numerical simulations.