Target audience: Advanced
Estimated reading time: 7'
Newsletter: Geometric Learning in Python
Principal Component Analysis (PCA) is essential for dimensionality and noise reduction, feature extraction, and anomaly detection. However, its effectiveness is limited by the strong assumption of linearity.
Principal Geodesic Analysis (PGA) addresses this limitation by extending PCA to handle non-linear data that lies on a lower-dimensional manifold.
Table of contents
Euclidean PCAWhat you will learn: How to implement principal geodesic analysis as extension of principal components analysis on a manifolds tangent space.
Notes:
- Environments: Python 3.10.10, Geomstats 2.7.0, Scikit-learn 1.4.2
- This article assumes that the reader is somewhat familiar with differential and tensor calculus [ref 1]. Please refer to our previous articles related to geometric learning [ref 2, 3, 4].
- Source code is available at Github.com/patnicolas/Data_Exploration/manifolds
- To enhance the readability of the algorithm implementations, we have omitted non-essential code elements like error checking, comments, exceptions, validation of class and method arguments, scoping qualifiers, and import statements.
Introduction
This article is the ninth installments of our ongoing series focused on geometric learning. As with previous articles, we utilize the Geomstats Python library [ref. 5] to implement concepts associated with geometric learning.
Note: Summaries of my earlier articles on this topic can be found in the Appendix
As a reminder, the primary goal of learning Riemannian geometry is to understand and analyze the properties of curved spaces that cannot be described adequately using Euclidean geometry alone.
This article revisits the widely used unsupervised learning technique, Principal Component Analysis (PCA), and its counterpart in non-Euclidean space, Principal Geodesic Analysis (PGA).
The content of this article is as follows:
- Brief recap of PCA
- Overview of key components of differential geometry
- Introduction to PCA on tangent space using the logarithmic map
- Implementation in Python using the Geomstats library
Principal components
Principal component analysis
Fig. 2 Illustration of a tangent vector and geodesic on a sphere
The mean point on the data point x[I] on the manifold can be defined as either a barycenter or a Frechet Mean [ref 6]. Our implementation in the next section relies on the Frechet mean.
Implementation
For the sake of simplicity, we illustrate the concept of applying PCA on manifold geodesics using a simple manifold, Hypersphere we introduced in a previous article, Differentiable Manifolds for Geometric Learning: Hypersphere
Setup
Let's encapsulate the evaluation of principal geodesics analysis in a class HyperspherePCA. We leverage the class HyperphereSpace [ref 7] its implementation of random generation of data points on the sphere.
ff from manifolds.hyperspherespace import HypersphereSpace
I import numpy as np
class HyperspherePCA(object):
def __init__(self):
self.hypersphere_space = HypersphereSpace(equip=True)
def sample(self, num_samples: int) -> np.array:
return self.hypersphere_space.sample(num_samples)
Euclidean PCA
First let's implement the traditional PCA algorithm for 3 dimension (features) data set using scikit-learn library with two methods:
- euclidean_pca_components to extract the 3 eigenvectors along the 3 eigenvalues
- euclidean_pca_transform to project the evaluation data onto 3 directions defined by the eigenvectors.
from sklearn.decomposition import PCA
@staticmethod
def euclidean_pca_components(data: np.array) -> (np.array, np.array):
num_components = 3
pca = PCA(num_components)
pca.fit(data)
return (pca.singular_values_, pca.components_)
@staticmethod
def euclidean_pca_transform(data: np.array) -> np.array:
num_components = 3
pca = PCA(num_components)
return pca.fit_transform(data
We compute the principal components on 256 random 3D data points on the hypersphere.
nu num_samples = 256 pca_hypersphere = HyperspherePCA() # Random data on Hypersphere data = pca_hypersphere.sample(num_samples) eigenvalues, components = HyperspherePCA.euclidean_pca_components(data) transformed = pca_hypersphere.euclidean_pca_transform(data) print(f'\nPrincipal components:\n{components}\nEigen values: {eigenvalues') print(f'\nTransformed data:\n{transformed}')
Output
Principal components:
[[ 0.63124842 -0.7752775 -0.02168467]
[ 0.68247094 0.54196625 0.49041411]
[ 0.36845467 0.32437229 -0.8712197 ]]
Eigen values: [9.84728751 9.0183337 8.65835924]
Important note: The 3 eigenvalues are similar because the input data is random.
Tangent PCA on hypersphere
The private method __tangent_pca computes principal components on the tangent plane using the logarithmic map. As detailed in the previous section, this implementation employs the Frechet mean as the base point on the manifold, which is the argument of the Geomstats method fit.
The method tangent_pca_components extracts the principal components computed using the logarithmic map. Finally the method tangent_pca_transform projects input data along the 3 principal components, similarly to its Euclidean counterpart, euclidean_pca_transform .
from geomstats.learning.pca import TangentPCA from geomstats.learning.frechet_mean import FrechetMean
def tangent_pca_components(self, data: np.array) -> np.array:
tgt_pca = self.__tangent_pca(data)
return tgt_pca.components_
def tangent_pca_transform(self, data: np.array) -> np.array:
tgt_pca = self.__tangent_pca(data)
tgt_pca.transform(data)
def __tangent_pca(self, data: np.array) -> TangentPCA:
sphere = self.hypersphere_space.space
tgt_pca = TangentPCA(sphere)
# We use the Frechet mean as center of data points on the hypersphere
mean = FrechetMean(sphere, method="default")
mean.fit(data)
# Frechet mean estimate
estimate = mean.estimate_
# Invoke Geomstats fitting method for PCA on tangent plane
tgt_pca.fit(data, base_point=estimate)
return tgt_pca
We evaluate the principal geodesic components on hypersphere using the same 256 points randomly generated on the hypersphere and compared with the components on the Euclidean space.
nu num_samples = 256 pca_hypersphere = HyperspherePCA() # Random data point generated on Hypersphere data = pca_hypersphere.sample(num_samples) _, components = HyperspherePCA.euclidean_pca_components(data) tangent_components = pca_hypersphere.tangent_pca_components(data) print(f'\nEuclidean PCA components:\n{components}\nTangent Space PCA components:\n{tangent_components}')
Output:
Euclidean PCA components:
[[ 0.63124842 -0.7752775 -0.02168467]
[ 0.68247094 0.54196625 0.49041411]
[ 0.36845467 0.32437229 -0.8712197 ]]
Tangent Space PCA components:
[[ 0.8202982 -0.45814582 0.34236423]
[ 0.45806882 0.16783651 -0.87292832]
[-0.34246724 -0.87288792 -0.34753831]]
References
--------------------------------------
Patrick Nicolas has over 25 years of experience in software and data engineering, architecture design and end-to-end deployment and support with extensive knowledge in machine learning.
He has been director of data engineering at Aideo Technologies since 2017 and he is the author of "Scala for Machine Learning", Packt Publishing ISBN 978-1-78712-238-3 and Geometric Learning in Python Newsletter on LinkedIn.
Patrick Nicolas has over 25 years of experience in software and data engineering, architecture design and end-to-end deployment and support with extensive knowledge in machine learning.
He has been director of data engineering at Aideo Technologies since 2017 and he is the author of "Scala for Machine Learning", Packt Publishing ISBN 978-1-78712-238-3 and Geometric Learning in Python Newsletter on LinkedIn.
Appendix
Here is the list of published articles related to geometric learning:
- Foundation of Geometric Learning introduces differential geometry as an applied to machine learning and its basic components.
- Differentiable Manifolds for Geometric Learning describes manifold components such as tangent vectors, geodesics with implementation in Python for Hypersphere using the Geomstats library.
- Intrinsic Representation in Geometric learning reviews the various coordinates system using extrinsic and intrinsic representation.
- Vector and Covector fields in Python describes vector and co-vector fields with Python implementation in 2 and 3-dimension spaces.
- Geometric Learning in Python: Vector Operators illustrates the differential operators, gradient, divergence, curl and laplacian using SymPy library.
- Functional Data Analysis in Python describes the key elements of non-linear functional data analysis to analysis curves, images, or functions in very high-dimensional spaces
- Riemann Metric & Connection for Geometric Learning reviews Riemannian metric tensor, Levi-Civita connection and parallel transport for hypersphere.
- Riemann Curvature in Python describes the intricacies of Riemannian metric curvature tensor and its implementation in Python using Geomstats library.
- K-means on Riemann Manifolds compares the implementation of k-means algorithm on Euclidean space using Scikit-learn and hypersphere using Geomstats
- Logistic Regression on Riemann Manifolds describes the implementation of the logistic regression on Hypersphere.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.