Feature extraction, dimensionality reduction and manifold learning

Dimensionality Reduction via Regression (DRR) is a manifold learning technique aimed at removing residual statistical dependence between PCA components due to dataset curvature. DRR predicts PCA coefficients from neighboring coefficients using multivariate regression, generalizing PPA. It advances dimensionality reduction methods by using curves instead of straight lines.

EPLS (Enhancing Population and Lifetime Sparsity) is an unsupervised feature learning algorithm designed for sparse representations in convolutional neural networks. It is meta-parameter free, simple, and fast.

HOCCA is a linear manifold learning technique that applies to datasets from the same source. It finds independent components in each dataset that are related across datasets, thus combining the goals of ICA and CCA.

KEMA extends SSMA by using kernel methods for better semantic alignments of multisource data.

KSNR is a feature extraction method that maximizes signal variance while minimizing noise variance in a reproducing kernel Hilbert space (RKHS). It provides noise-free features for dimensionality reduction, outperforming kPCA in correlated noise scenarios.

OKECA is a kernel feature extraction method based on entropy estimation in Hilbert spaces. It provides sparse and compact results, useful for data visualization and dimensionality reduction.

Principal Polynomial Analysis (PPA) is a manifold learning technique that generalizes PCA by using principal polynomials to capture nonlinear data patterns. It improves PCA’s energy compaction ability, reducing dimensionality reduction errors. PPA defines a manifold-dependent metric that generalizes Mahalanobis distance for curved manifolds.

RBIG is an invertible multivariate Gaussianization transform that uses univariate histogram Gaussianization and multivariate rotation. This method is useful for multivariate PDF estimation and associated applications.

The rotated complex kernel PCA (ROCK-PCA) works in reproducing kernel Hilbert spaces to account for nonlinear processes, operates in the complex domain to handle both spatial and temporal features and time-lagged correlations. It adds an extra rotation for improved flexibility and physical consistency, providing an explicitly resolved spatio-temporal decomposition of Earth and climate data cubes.

SIMFEAT is a toolbox that includes linear and kernel feature extraction methods. Linear methods: PCA, MNF, CCA, PLS, OPLS. Kernel methods: KPCA, KMNF, KCCA, KPLS, KOPLS, KECA.

SPCA is an invertible manifold learning technique that generalizes PCA by using nonparametric principal curves instead of straight lines. It includes multivariate histogram equalization to fulfill either NonLinear ICA or optimal Vector Quantization.

SSKPLS utilizes probabilistic cluster kernels for nonlinear feature extraction. It builds kernel functions from data, outperforming standard kernel functions and information theoretic kernels like Fisher and mutual information kernels.

The SSMA Toolbox is a MATLAB tool for semisupervised manifold alignment of data without corresponding pairs, requiring only a small set of labeled samples in each domain.