Fast and unique Tucker decompositions via multiway blind source separation

Open access


A multiway blind source separation (MBSS) method is developed to decompose large-scale tensor (multiway array) data. Benefitting from all kinds of well-established constrained low-rank matrix factorization methods, MBSS is quite flexible and able to extract unique and interpretable components with physical meaning. The multilinear structure of Tucker and the essential uniqueness of BSS methods allow MBSS to estimate each component matrix separately from an unfolding matrix in each mode. Consequently, alternating least squares (ALS) iterations, which are considered as the workhorse for tensor decompositions, can be avoided and various robust and efficient dimensionality reduction methods can be easily incorporated to pre-process the data, which makes MBSS extremely fast, especially for large-scale problems. Identification and uniqueness conditions are also discussed. Two practical issues dimensionality reduction and estimation of number of components are also addressed based on sparse and random fibers sampling. Extensive simulations confirmed the validity, flexibility, and high efficiency of the proposed method. We also demonstrated by simulations that the MBSS approach can successfully extract desired components while most existing algorithms may fail for ill-conditioned and large-scale problems.

[1] F. Hitchcock, “Multiple invariants and generalized rank of a p-way matrix or tensor”, J. Mathematics and Physics 7, 39-79 (1927).

[2] L. Tucker, “Some mathematical notes on three-mode factor analysis”, Psychometrika 31 (3), 279-311 (1966).

[3] L. De Lathauwer, B. De Moor, and J. Vandewalle, “A multilinear singular value decomposition”, SIAM J. on Matrix Analysisand Applications 21, 1253-1278 (2000).

[4] T.G. Kolda and B.W. Bader, “Tensor decompositions and applications”, SIAM Review 51 (3), 455-500 (2009).

[5] A. Cichocki, R. Zdunek, A.-H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to ExploratoryMulti-way Data Analysis and Blind Source Separation, Wiley, Chichester, 2009.

[6] A. Cichocki, “Tensors decompositions: new concepts for brain data analysis?”, J. Control, Measurement, and System Integration(SICE) 7, 507-517 (2011).

[7] L. De Lathauwer, B. De Moor, and J. Vandewalle, “On the best rank-1 and rank-(R1,R2, ...,Rn) approximation of higherorder tensors”, SIAM J. on Matrix Analysis and Applications 21 (4), 1324-1342 (2000).

[8] F. Miwakeichi, E. Martnez-Montes, P. Valds-Sosa, N. Nishiyama, H. Mizuhara, and Y. Yamaguchi, “Decomposing EEG data into space-time-frequency components using parallel factor analysis”, NeuroImage 22 (3), 1035-1045 (2004).

[9] Z. He, A. Cichocki, S. Xie, and K. Choi, “Detecting the number of clusters in n-way probabilistic clustering”, IEEE Trans.on Pattern Analysis and Machine Intelligence 32 (11), 2006- 2021 (2010).

[10] H.A. Phan and A. Cichocki, “Tensor decompositions for feature extraction and classification of high dimensional datasets”, Nonlinear Theory and Its Applications, IEICE 1 (1), 37-68 (2010).

[11] R. Bro, “Multi-way analysis in the food industry: Models, algorithms & applications”, Food Technology 309,−Bro.pdf, (1998).

[12] A.H. Andersen and W.S. Rayens, “Structure-seeking multilinear methods for the analysis of fMRI data”, NeuroImage 22 (2), 728-739 (2004).

[13] E. Mart´inez-Montes, P.A. Vald´es-Sosa, F. Miwakeichi, R. Goldman, and M. Cohen, “Concurrent EEG/fMRI analysis by multiway partial least squares”, NeuroImage 22 (3), 1023- 1034 (2004).

[14] E. Acar, C. Aykut-Bingol, H. Bingol, R. Bro, and B. Yener, “Multiway analysis of epilepsy tensors”, Bioinformatics 23 (13), i10-i18 (2007).

[15] M. De Vos, A. Vergult, and L. De Lathauwer, “Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone”, NeuroImage 37 (3), 844-854 (2007).

[16] M. Mørup, L.K. Hansen, and S.M. Arnfred, “Algorithms for sparse nonnegative Tucker decompositions”, Neural Computation 20 (8), 2112-2131 (2008).

[17] L. De Lathauwer, “A survey of tensor methods”, IEEE Int.Symposiumon Circuits and Systems (ISCAS) 1, 2773-2776 (2009).

[18] V. De Silva and L.-H. Lim, “Tensor rank and the ill-posedness of the best low-rank approximation problem”, SIAM J. on MatrixAnalysis and Applications 30 (3), 1084-1127 (2008).

[19] S. Weiland and F. Van Belzen, “Singular value decompositions and low rank approximations of tensors”, IEEE Trans.on Signal Processing 58 (3), 1171-1182 (2010).

[20] L. De Lathauwer, “A link between the canonical decomposition in multilinear algebra and simultaneous matrix diagonalization”, SIAM J. on Matrix Analysis and Applications 28 (3), 642-666 (2006).

[21] C.F. Beckmann and S.M. Smith, “Tensorial extensions of independent component analysis for multisubject fMRI analysis”, NeuroImage 25 (1), 294-311 (2005).

[22] M. De Vos, L. De Lathauwer, and S. Van Huffel, “Algorithm for imposing SOBI-type constraints on the CP model”, IEEEInt. Symp. Circuits and Systems (ISCAS)., 2008, 1344-1347.

[23] M. Vasilescu and D. Terzopoulos, “Multilinear independent components analysis”, IEEE Computer Society Conf. on ComputerVision and Pattern Recognition 1, 547-553 (2005).

[24] S. Unkel, A. Hannachi, N. Trendafilov, and I. Jolliffe, “Independent component analysis for three-way data with an application from atmospheric science”, J. Agricultural, Biological,and Environmental Statistics 16, 319-338 (2011).

[25] A. Bell and T. Sejnowski, “An information-maximization approach to blind separation and blind deconvolution”, NeuralComputation 7, 1129-1159 (1995).

[26] A. Hyvarinen, J. Karhunen, and E. Oja, Independent ComponentAnalysis, Wiley, New York, 2001.

[27] A. Cichocki and S. Amari, Adaptive Blind Signal and ImageProcessing: Learning Algorithms and Applications, John Wiley & Sons, London, 2002.

[28] A. Karfoul, L. Albera, and L. De Lathauwer, “Iterative methods for the canonical decomposition of multi-way arrays: Application to blind underdetermined mixture identification”, SignalProcessing 91 (8), 1789-1802 (2011).

[29] M. Zibulevsky and B. A. Pearlmutter, “Blind source separation by sparse decomposition in a signal dictionary”, NeuralComputation 13 (4), 863-882 (2001).

[30] P. Georgiev, F. Theis, and A. Cichocki, “Sparse component analysis and blind source separation of underdetermined mixtures”, IEEE Trans. on Neural Networks 16 (4), 992-996 (2005).

[31] D.D. Lee and H.S. Seung, “Learning the parts of objects by non-negative matrix factorization”, Nature 401 (6755), 788- 791 (1999).

[32] B.W. Bader and T.G. Kolda, “MATLAB tensor toolbox version 2.5”,, (2012).

[33] G. Zhou and A. Cichocki, “Canonical polyadic decomposition based on a single mode blind source separation”, IEEE SignalProcessing Letters 19 (8), 523-526 (2012).

[34] X. Feng and Z. Zhang, “The rank of a random matrix”, AppliedMathematics and Computation 185 (1), 689-694 (2007).

[35] F.M. Fisher, The Identification Problem in Econometrics, McGraw-Hill, New York, 1966.

[36] M. De Vos, D. Nion, S. Van Huffel, and L. De Lathauwer, “A combination of parallel factor and independent component analysis”, Tech. Rep.,−cp08.pdf (2008).

[37] D. Langers, “Unbiased group-level statistical assessment of independent component maps by means of automated retrospective matching”, Human Brain Mapping 31, 727-742 (2010).

[38] A. Cichocki, “Generalized component analysis and blind source separation methods for analyzing mulitchannel brain signals”, Statistical and Process Models for Cognitive Neuroscienceand Aging 1, 201-272 (2007).

[39] Z. Lin, M. Chen, and Y. Ma, “The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices”, ArXiv e-prints, 1009.5055 (2010).

[40] P. Drineas, R. Kannan, and M.W. Mahoney, “Fast Monte Carlo algorithms for matrices II: computing a low-rank approximation to a matrix”, SIAM J. on Computing 36 (1), 158-183 (2006).

[41] M.W. Mahoney and P. Drineas, “CUR matrix decompositions for improved data analysis”, Proc. Nat. Ac. Sciences 106 (3), 697-702 (2009).

[42] C.F. Caiafa and A. Cichocki, “Generalizing the column-row matrix decomposition to multi-way arrays”, Linear Algebra andIts Applications 433 (3), 557-573 (2010).

[43] E. Ceulemans and H.A.L. Kiers, “Selecting among three-mode principal component models of different types and complexities: a numerical convex hull based method”, British J. Mathematicaland Statistical Psychology 59 (1), 133-150 (2006).

[44] H.A.L. Kiers and A. der Kinderen, “A fast method for choosing the numbers of components in tucker3 analysis”, BritishJ. Mathematical and Statistical Psychology 56 (1), 119-125 (2003).

[45] M.O. Ulfarsson and V. Solo, “Dimension estimation in noisy pca with sure and random matrix theory”, IEEE Trans. onSignal Processing 56 (12), 5804-5816 (2008).

[46] A. Cichocki, S. Amari, and K. Siwek, “ICALAB toolbox”, (2007).

[47] Z. Koldovsky, P. Tichavsky, and E. Oja, “Efficient variant ofalgorithm FastICA for independent component analysis attaining the Cramer-Rao lower bound”, IEEE Trans. on Neural Networks 17 (5), 1265-1277 (2006).

[48] A. Belouchrani, K. AbedMeraim, and J.F. Cardoso, “A blind source separation technique using second-order statistics”, IEEE Trans. on Signal Processing 45 (2), 434-444 (1997).

[49] C.A. Andersson and R. Bro, “The N-way toolbox for MATLAB”, (2000).

[50] A.H. Phan and A. Cichocki, “Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification”, Neurocomputing 74 (11), 1956-1969 (2011).

[51] R. Zdunek, H.A. Phan, and A. Cichocki, “Damped Newton iterations for nonnegative matrix factorization”, Australian J. IntelligentInformation Processing Systems 12 (1), 16-22 (2010).

[52] G. Zhou, A. Cichocki, and S. Xie, “Fast nonnegative matrix/ tensor factorization based on low-rank approximation”, IEEE Trans. on Signal Processing 60 (6), 2928-2940 (2012).

[53] S.A. Nene, S.K. Nayar, and H. Murase, “Columbia object image library (COIL-20)”, http://www.cs.columbia. edu/ CAVE/software/softlib/coil-20.php (1996).

[54] L. Van Der Maaten and C. Detection, “Visualizing data using t-SNE”, J. Machine Learning Research 9, 2579-2605 (2008).

[55] Y. Yang, D. Xu, and F. Nie, “Image clustering using local discriminant models and global integration”, IEEE Trans. onImage Processing 19 (10), 2761-2773 (2010).

[56] B.Z.Allison, D.J. McFarland, G. Schalk, S.D. Zheng, M. Jackson, and J.R. Wolpaw, “Towards an independent braincomputer interface using steady state visual evoked potentials”, Clinical Neurophysiology 119 (2), 399-408 (2008).

[57] Y. Zhang, G. Zhou, Q. Zhao, A. Onishi, J. Jin, X. Wang, and A. Cichocki, “Multiway canonical correlation analysis for frequency components recognition in SSVEP-based BCIs”, Neural Information Processing, Springer, Berlin, 2011.

[58] X. Gao, D. Xu, M. Cheng, and S. Gao, “A BCI-based environmental controller for the motion-disabled”, IEEE Trans. onNeural Systems and Rehabilitation Engineering 11 (2), 137- 140 (2003).

[59] Z. Wu and D. Yao, “Frequency detection with stability coefficient for steady-state visual evoked potential (SSVEP)-based BCIs”, J. Neural Engineering 5 (1), 36 (2008).

Bulletin of the Polish Academy of Sciences Technical Sciences

The Journal of Polish Academy of Sciences

Journal Information

IMPACT FACTOR 2016: 1.156
5-year IMPACT FACTOR: 1.238

CiteScore 2016: 1.50

SCImago Journal Rank (SJR) 2016: 0.457
Source Normalized Impact per Paper (SNIP) 2016: 1.239


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 148 148 20
PDF Downloads 41 41 6