Weighted low rank approximation : Algorithms and applications

In order to find more sophisticated trends in data, potential correlations between larger and larger groups of variables must be considered. Unfortunately, the number of such correlations generally increases exponentially with the number of input variables and, as a result, brute force approaches become unfeasible. So, the data needs to be simplified sufficiently. Yet, the data may not be oversimplified. A method that is widely used for this purpose is to first cast the data as a matrix and the compute a low rank matrix approximation. The tight equivalences between the Weighted Low Rank Approximation (WLRA) problem and the Total Least Squares (TLS) problem are explored. Despite the seemingly different problem formulations of WLRA and TLS, it is shown that both methods can be reduced to the same mathematical kernel problem, i.e. finding the closest (in a certain sense) weighted low rank matrix approximation where the weight is derived from the distribution of the errors in the data. Different solution approaches, as used in WLRA and TLS, are discussed. In particular, we discuss the Null space parameterized WLRA (NullWLRA the Maximum Likelihood Principal Analysis (MLPCA), the Elementwise Weighted-TLS (EW-TLS) and the generalized TLS (GTLS) methods. It is shown that these four approaches tackle an equivalent weighted low rank approximation problem, but different algorithms are used to come up with the best approximation matrix. The WLRA problem is studied in the application field of chemometrics. In chemometrics, existing approaches to come up with the best weighted low rank matrix approximation are the well-known Principal Component Analysis (PCA) method and the MLPCA method. The use of TLS-like algorithms in chemometrics is discussed. An adapted version of the EW-TLS algorithms is developed to solve the WLRA problem for data matrices in chemometrics with more columns than rows and with only row-wise correlated measurements errors. It is shown that this new algorithm is a good alternative for the existing methods used in chemometrics. The WLRA approach is extended towards linearly structured matrices. The Hankel structure is investigated since this is one of the most frequently occurring structures in signal processing applications. We study the cases of scalar Hankel matrices and block-row Hankel matrices. For both cases, new algorithms are presented in order to solve these structured WLRA problems. These algorithms can handle structured WLRA problems with rank reductions larger than one, while most existing algorithms from the literature can only handle rank-one reductions problems. By means of simulation experiments the improved statistical accuracy of the proposed algorithms compared to known algorithms from the literature is confirmed. The structured WLRA problem is studied in the field of recovering the vertices of a planar polygon from its measured complex moments. In the literature, the use of the structured TLS approach to solve this shape-from-moments problem has not been discussed. Therefore, the potential and limitations of the structured TLS algorithm and existing algorithms to solve this reconstruction problem are discussed.