2.2.5. A second example about variables representing the same concept

<< Click to Display Table of Contents >>

Navigation:  2. Introduction to the PCA techniques > 2.2. Distance definition >

2.2.5. A second example about variables representing the same concept

 

Let’s take another example: We have a dataset with 3 columns. These 3 columns are:
 

The Age expressed in Days                (AgeD)
 

The Age expressed in Months        (AgeM)
 

The Size                                 (Size)

 
 
Using these 3 axises, we can define this Euclidian Distance:

 

clip0012

 

(Note that we are using « normalized » distance)

 

We can obtain the “age expressed in days”(AgeD) of an individual by multiplying by 31 the age of the same individual expressed in months. When we are using this relationship inside the equation of the Euclidean distance we arrive to this final equation.

 

clip0013

 
There is something very wrong about this final equation: We are changing the normalization factor. The « weight » given to the « age concept » is wrongly multiplied by more than 900. The « normalization » of the variables is lost when two variables (here: the “age expressed in days” and the “age expressed in months”) are representing the same concept.

 

Let’s represent graphically the “age expressed in days”(AgeD) and the “age expressed in months”(AgeM):

 

STARDU~1_img69

 

Each cross here is an individual. For example:
 

an individual that is 31 days old is, indeed, one month old (AgeD=31; AgeM=1). number1

an individual that is 63 days old is, indeed, 2 monthes old (AgeD=63; AgeM=2). number2

 

 

You may remember this chart from the previous section about “dimensionality reduction”:

 

STARDU~1_img27

 

On this chart we computed the 2 PCA axises in order to reduce the dimension. We will do the same on this new population:

 

 

STARDU~1_img72

 

 

We obtain 2 PCA axises named “PCA1” and “PCA3”. If we compute the coordinates of each individual inside the new coordinate system defined by the “PCA1” and “PCA3” axises. We will notice that:
 

OLD        individuals have a HIGH value along the “PCA1” axis
 

YOUNG individuals have a LOW   value along the “PCA1” axis.

 
 

So the PCA1 axis represents the “age” concept. The PCA3 direction represents the small error in converting the age from months to days. The values along the PCA3 are completely uninteresting so we will simply never use them.

 

 
So, instead of defining the distance on the original axises: “age expressed in days”(AgeD) , “age expressed in months”(AgeM) and “size”(SIZE):

 

clip0016

 

… we will now express the distance inside the PCA axises:
 

PCA1 that represents the “age” concept
 

PCA2 that represents the “size” concept
 

PCA3 that represents the small error in converting the age from months to days.

 

 
This gives us:

 

clip0017

 

Note that we did not include inside this new definition of the Distance, the distance along the « PCA3 » direction because it’s completely irrelevant. You can see that it is irrelevant because the range of values in the PCA direction is very small. This new definition of the distance is the best one possible. You can see the range along the « PCA3 » axis here:

 

STARDU~1_img77

 
 

This range is so small that it is totally useless and does not represent anything: it’s noise. So, it’s important to know the range of value along each PCA direction because you must drop from the distance-definition the PCA directions that have small variance. StarDust automatically displays the range of values along all PCA axis: In this graphic (from StarDust), we see that the range along the PCA8 axis is very small:

 

clip0014

 

 
So, a good distance definition should only include terms from PCA1 to PCA7 and NOT PCA8: For example:

clip0015