Let’s consider another example. We still want to reduce the dimension of the 3D points to only 2D. Please note that there are two different families of points: the crosses and the circles.
Let’s project all the 3D points on the plane defined by the PCA1 and PCA2 axises. The points after projection are represented in Blue. The projection direction in itself is in green:
The PCA is working fine and, after projection, we can still differentiate the two families of points: the "crosses" are far away from the "circles". The PCA allows us to « visually » find segments inside the population: you can easily « see the segments » after a projection on the PCA axises.
What happens if we use some other directions than the PCA directions to perform dimensionality reduction? Let’s project all the 3D points on a randomly chosen plane. The points after projection are in Blue. The projection direction in itself is in green. Note that the green lines are very « long »: it means that “this is a very bad projection” because the position of the points before and after projection is very different.
After projection, we cannot differentiate the two families of points anymore: the "crosses" are now mixed with the "circles":
This is a really bad « dimensionality reduction » because we lost so much information that we are not able to see any segments anymore.