Question

[Linear Algebra] Can a transformation matrix have a rank higher than the number of its rows/cols?

It seems reasonable to say that is impossible for linear transformations, but what about none-linear ones?

Answer 1

a "transformation" matrix might be confusing terminology, I mean any matrix in general that's just an interpretation.

Answer 2

No. The rank-nullity theorem says that for a matrix \(\mathbf A\), the rank of \(\mathbf A\) and the nullity of \(\mathbf A\) (or the dimension of the nullspace of \(\mathbf A\)) add up to the number of columns of \(\mathbf A\).

So consider some matrix \(\mathbf A\) with \(n\) columns, and suppose that the rank is larger than \(n\). Then
\[\mathrm{null}\mathbf A=n-\mathrm{rank}\mathbf A<0\]This is saying that the dimension of the nullspace of \(\mathbf A\) is negative, or that there's a negative number of vectors in the basis for the nullspace. But this is absurd, so it's not possible.

Answer 3

Oh! Hmm that certainly makes sense, I deduced the same for linear transformations. I just wasn't sure if the theorem was still valid for none-linear ones.

Answer 4

@HolsterEmission

Answer 5

But that is beyond "linear" algebra's scope ahaha

Answer 6

Nonlinear transformations can't always be represented as matrices. There are some (translations, rotations) that can, but those are special cases.

For example, there is no matrix \(\mathbf T\in\mathbb R^{n\times n}\) such that for any vector \(\mathbf v\in\mathbb R^n\), the product \(\mathbf{Tv}\) results in a translation by some non-zero vector \(\mathbf a\). However, we can find an augmented matrix \(\overline{\mathbf T}\in\mathbb R^{(n+1)\times(n+1)}\) that acts on the augmented version of \(\overline{\mathbf v}\in\mathbb R^{n+1}\) that can produces something you can work with.

Let \(\mathbf v=\begin{bmatrix}v_1&\cdots&v_n\end{bmatrix}^\intercal\), and suppose \(\mathbf a=\begin{bmatrix}a_1&\cdots&a_n\end{bmatrix}^\intercal\neq\mathbf0\). We can transform \(\overline{\mathbf v}=\begin{bmatrix}v_1&\cdots&v_n&1\end{bmatrix}^\intercal\) into \(\overline{\mathbf v+\mathbf a}=\begin{bmatrix}v_1+a_1&\cdots&v_n+a_n&1\end{bmatrix}^\intercal\) with the matrix
\[\overline{\mathbf T}=\begin{bmatrix}
1&0&0&\cdots&0&0&a_1\\
0&1&0&\cdots&0&0&a_2\\
0&0&1&\cdots&0&0&a_3\\
\vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots\\
0&0&0&\cdots&1&0&a_{n-1}\\
0&0&0&\cdots&0&1&a_n\\
0&0&0&\cdots&0&0&1
\end{bmatrix}\]

Answer 7

Oh hmm. I don't really get why this augmented version allows for non-linear transformations just yet, but those augmented non-linear transformations still cannot expand beyond the dimension of their original scope right? it would be a very weird transformation to be able to generate a plane out of a line.

Answer 8

Well, let's carry out the matrix product for a simple case, say \(\overline{\mathbf T}\in\mathbb R^3\). This transformation gives us a means of translating the vector \(\mathbf v\) to \(\mathbf v+\mathbf a\).
\[\overline{\mathbf T}\overline{\mathbf v}=\begin{bmatrix}1&0&a_1\\0&1&a_2\\0&0&1\end{bmatrix}\begin{bmatrix}v_1\\v_2\\1\end{bmatrix}=\begin{bmatrix}v_1+a_1\\v_2+a_2\\1\end{bmatrix}=\overline{\mathbf v+\mathbf a}\]That last component is really just a placeholder. This is the kind of method a computer would use to view a nonlinear transformation as a standard one.

Answer 9

hmmm, but what is non-linear about this? is the translation of a vector to some other vector non-linear?

Answer 10

This transformation is linear, but it required adding another dimension to \(\mathbf T\) for it to work. Translations are nonlinear transformations but can be viewed as linear ones in higher dimensions. There is no \(\mathbf T\) that can give \(\mathbf{Tv}=\mathbf v+\mathbf a\), hence the need for \(\overline{\mathbf T}\).

Answer 11

Oh yeah that makes sense, since translations defy the rule of staying at the origin, but in higher dimensions we can make that work for the lower planes