Something is unclear for me about the Nullspace matrix. When we generate the rref(A), we'll have R=[IF] but the I and the F sometimes overlap. When we generated the special solutions from the U matrix, we were watching our pivot and free columns and assigning number to the exact xi variables that were represented by the particular free column. After the rref step and generating R=[IF], they reordered the columns if they were overlapping. But this column reordering is changing the variable itself, the free column will represent a totally different variable because of the change. Is this valid??

Question

anonymous · Answer

I've read the referring pages from the textbook also and there is no mention of why can we do this reordering, it just pure and simply reorders there too. Isn't this reordering totally changing the vectors for the Nullspace?

datanewb · Answer

I think you are correct.  When the I and the F overlap, that is the pivot columns are mixed with the free columns, you cannot rearrange the columns without changing the matrix, R, and thus solutions to equations such as Rx = 0. As far as I can tell, R = IF is a simplified way to express this, and during the corresponding lecture I think I remember that Dr. Strang wrote the 'I' and the 'F' so that the letters themselves were overlapping.

anonymous · Answer

I remember the overlapping I and F letters on the blackboard. My point is: there is no worked example for using the N=[-F] stuff when this happens... no example on the videos nor in the book. The book only mentions that if these overlap than we could just switch columns. I tried to do an example myself and the results are not correct this way. Something else probably is missing but is not discussed more briefly.

anonymous · Answer

I'm not entirely sure what you mean by overlapping and pivot matrix, but if you take a matrix and move an entire column or an entire row you do not change the solution to the matrix.

datanewb · Answer

@jsmerson, that does not compute. Start with the identity matrix, I. 
[1 0]
[0 1]

Move either an entire row or an entire column to get matrix P.
[0 1]
[1 0].

P does not equal I.

anonymous · Answer

@datanewb This is true, p doesn't equal I, but the relationship between the two vectors is maintained. Swapping rows or columns are called elementary row/column operations and these do not change the solution to the matrix. See this website for a list of these operations: http://mathworld.wolfram.com/ElementaryRowandColumnOperations.html

anonymous · Answer

So how can we use the N=[-F] method to solve the equations if the pivot columns and the free columns are overlapping? I've tried to write the solutions using both the N=[-F] method and the regular way. The resultant vectors are not the same so I tried to see if the two solutions are equal(linear combinations of the other method's solutions) and they are not. The column swapping ruins the solutions for the N=[-F] method, at least for me. How do you proceed  correctly in this case?

anonymous · Answer

can you give me a link for the N=[-F] method you are talking about? I didn't take the MIT course...

also if you give me a matrix I can write out the steps for you. I'm having a hard time putting it into understandable terms without an example

anonymous · Answer

This is the short lecture note from the MIT course, they mention the N=[-F] idea at the bottom of the second page. http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/ax-b-and-the-four-subspaces/solving-ax-0-pivot-variables-special-solutions/MIT18_06SCF11_Ses1.7sum.pdf They actually use a matrix with overlapping I and F parts but don't carry out the solution with the N=[-F] method. The book only tells that we can just exchange the columns. But if I do that, i get a totally different result... Traditional method: $$\left(\begin{matrix}-2 \ 1 \0 \0 \end{matrix}\right) \left(\begin{matrix}2 \ 0 \ -2 \1\end{matrix}\right)$$ N=[-F] method with exchanging columns: $$F=\left[\begin{matrix}2 & -2 \ 0 & 2\end{matrix}\right]; Solutions:N=\left[\begin{matrix}-2 & -2 \ 0 & -2 \ 1 & 0 \ 0 & 1 \end{matrix}\right]$$ So the row exchange affected the N matrix and these are not correct results, the second and third rows are exchanged. OK, if I exchange these rows than it's correct, but there is no mention in the course or the Strang book that we have to fiddle around with the N matrix, they just say that if we have overlapping I and F parts in the rref, we just exchange the columns and carry on. But my point is ... this is wrong, we would have to do a row exchange because of the previous column exchange. I would do this by multiplying N' (N transpose) by the inverse of the permutation matrix used to exchange columns. Transpose the result back and it's Ok. But the lecture and the book just tells to exchange columns and then we can use the N matrix method and nothing else. Sorry for being a bit circular :)

datanewb · Answer

I've rewatched the lecture in question, Lecture 7, and I think this is one thing that Strang did not explain as clearly as possible.  He should have reminded us (more often) that column operations DO CHANGE the row space and the nullspace of A, or solutions to Ax = 0!  That is why he only performs row operations in the algorithm rref(A) = R. $$R = \left[\begin{matrix}I & F \ 0 & 0\end{matrix}\right] $$ & $$N = \left[\begin{matrix}-F \ I\end{matrix}\right]$$ are just a shorthand.  Sometimes I and F overlap.  You cannot exchange columns of R without changing the solution to Rx = 0.  Sometimes the pivot columns are the first columns of A, but if they are not then I and F overlap.

Notice conversely that the row operations performed on A to get R DO change the column space, such that the columns of A are NOT linear combinations of R! (Similarly the left nullspace, which you learn about later, of R does not equal that of A.)

datanewb · Answer

Edit^ [i]That is why[/i] he only performs row operations in the algorithm rref(A) = R.
Should read:
It is for a similar reason...

anonymous · Answer

So how would you make the
$$N=\left[\begin{matrix}-F \ I\end{matrix}\right]$$
when they overlap in the R? The book and the video says you just exchange columns. That is the weird part.

datanewb · Answer

I rewatched the video, and I was under the impression that $$N = \left[\begin{matrix}-F\I\end{matrix}\right]$$ was a representation what the null matrix would look like with the pivot columns and free columns grouped together.  In no way did I feel that he was still manipulating a matrix R.

See the attachment jpg which is a screen shot where he grouped the pivot and free columns together from a matrix R in which they overlapped during one during lecture 7.  As you can clearly see, he didn't even write the matrix brackets around them because that would not be valid!

You were right all along!  When you perform column operations on R, you almost always change the solution to Rx = b (including the case where b = 0).

anonymous · Answer

Yes, that's what I meant.
But he just says we'll reorder and go on... I of course searched for this in the book, because the book has more details on everything. Well, not on this one. But the book also tells us that the reordering is OK. How can that be???
And of course the example below it is not an example for the overlapping case -.-

So both the video and the book says reordering columns does the job. But that's my problem... how? I mean it doesn't seem to be correct if I do it, but they keep saying it even in the book...

datanewb · Answer

To make it perfectly clear, the highlighted lines in your attached image could/should read "The idea is still true if the pivot columns are mixed in with the free columns.  Then I and F are mixed together.  You can still see the -F mixed with I in the solutions."