Question

What is the geometrical interpretation of the (A'A)^-1 within the projection formula:

P=A(A'A)^-1A' ?

Answer 1

I don't think there is any special geometrical interpretation of the \((A^TA)^{-1}\), which is just a symmetric coefficient matrix containing \(n\) independent vectors of \(n\)-dimension in \(R^n\). Obvious to point out, it's the application of \(P\) on \(b\) that has a geometrical meaning, which of course is the projection of the line \(b\) on to the \(A\) subspace.