$\begingroup$@whuber: I am not sure, but it looks like OP is not asking about centering/scaling the data matrix, but about centering/scaling PCA eigenvectors (probably in order to project the data onto them).$\endgroup$
– amoebaDec 12 '15 at 19:16

1

$\begingroup$@amoeba That would be such a strange thing to do that I didn't catch it, but indeed that seems to be what's going on here. It may come down to what an "explaining vector" is intended to mean.$\endgroup$
– whuber♦Dec 12 '15 at 21:23

1

$\begingroup$@amoeba Yes that's it! As far as they explain how the data is projected (feel free to correct me if I'm wrong).$\endgroup$
– ThePassengerDec 13 '15 at 20:33

2 Answers
2

Principal component analysis (PCA) in statistics commonly uses either the covariance matrix $\bf{C}$ or correlation matrix $\bf{R}$ for the variables (attributes). Therefore, for your problem, you would generate the $3 \times 3$ correlation matrix, and then perform eigendecomposition on $\bf{R}$. This is called solving the symmetric eigenvalue problem. The eigenvalues from $\bf{R}$ will essentially explain how many variables are correlated with one another, whereas, if the 3 variables are orthogonal (zero correlation) the eigenvalues should each be close to one. After eigendecomposition of $\bf{R}$, you can determine the determinant as $|\bf{R}|$$=\prod_j \lambda_j$, and the closer $|\bf{R}|$ is to unity the closer $\bf{R}$ is to the identity matrix $\bf{I}$ $\rightarrow$ zeroes in the off-diagonals of $\bf{R}$ and ones on the diagonal.

Without going into detail on the notation for the loadings and the PC scores, the loading matrix $\bf{L}$ will be a $3 \times 3$ matrix, which will the reveal correlation between each of the three variables with the 3 (orthogonal) principal components. I have always used 0.55 as an indicator for high loading. Sometimes variables can also load appreciably (rather than mostly) on more than one PC, and to decrease this tendency you can use a varimax orthogonal rotation -- which will make the variables load mostly on one component. (Computationally, there is also no distinction between a positive or negative sign of each loading, because the sign can change with the algorithm used. Therefore positive loading vs. negative should never be looked at as which sign is correct because you always have to compare signs using different packages).

Lastly, you will generate a $6 \times 3$ matrix $\bf{F}$ of PC scores, which represent you original data points in PC space. The 3 columns of PCs have zero correlation between them and by definition are standard normal distributed $\cal{N}(0,1)$ with mean zero and variance (s.d.) one.

Regarding normalization, the determination of $\bf{R}$ includes mean-zero standardization of your 3 variables, so skewness will be retained if present. Normalization will only rescale to range [0,1], whereas mean-zero standardization will result in a mean of zero for each variable, but not necessarily a variance of unity -- especially if you have skewness. If you want purely standard normal transforms, then calculate the van der Waerden scores for your data points within each variable.

$\begingroup$"They're already centered if your data is centered" - what would that mean? If the data matrix is centered, eigenvectors don't have to have zero mean. But of course I agree that it's nonsensical to "center" them.$\endgroup$
– amoebaDec 4 '17 at 21:01