Question

Linear Algebra/Linear Regression:
The Hat matrix is defined as \(H=X(X^TX)^{-1}X^T\). Messing with it a bit I found that it was equal to the identity matrix o_O? Can you show me where I went wrong?

Answer 1

It shouldn't be though since this matrix is significant for linear regression. (I mean why would they define such a long expression?).

\[X(X^TX)^{-1}X^T\]=\(X(X^{-1}(X^T)^{-1})X^T\)\[=X(X^{-1}(X^{-1})^{T})X^T\]\[=XX^{-1}(X^{-1})^TX^T\]\[=I(X^{-1})^TX^T\]\[=I(XX^{-1})^T\]\[=II=I\]

Answer 2

Took a little research, but you're correct. However, if \(X\) does not have an inverse, this does not result in the identity.

Answer 3

See here for a little more information http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse

Answer 4

o_o interesting. Hmm lol this Hat matrix is a lot more special than I thought lol

Answer 5

Same here.