Question

Re() = cos(angle) * ||a|| * ||b||... see: https://en.wikipedia.org/wiki/Dot_product#Complex_vectors and http://mathb.in/86533 for my proof. Is there a similar formula for the imaginary part? e.g. Im()... I have a feeling it might be equal to sin(angle) * ||a|| * ||b||

Answer 1

Not sure, but this might help. On the left I'm writing complex numbers but on the right I'm writing 2D vectors.
\[z^* w =\vec z \cdot \vec w + i|\vec z \times \vec w| \]

Answer 2

could you elaborate what that means?

Answer 3

I assume z* means the hermitian transpose of z?

Answer 4

also, does the x mean cross product?

Answer 5

I would like to apply this to vectors of any dimension

Answer 6

Nah that's not the Hermitian conjugate, I'm just talking about the individual complex numbers themselves. But these are what constitute the vectors you're using here and I think if you understand how the components combine you'll better understand how to use them.

So here, I'll write it out explicitly since it's not much trouble.

As complex numbers:
\[z=x+iy\]\[w=u+iv\]\[z^*w = (x-iy)(u+iv)=(xu+yv)+i(xv-yu)\]

Now we can also write them as vectors: (and yeah that's the cross product)
\[\vec z= (x,y)\]\[\vec w = (u,v)\]\[\vec z \cdot \vec w = xu+yv\]\[|\vec z \times \vec w| = xv-yu\]

Answer 7

ah I understand

Answer 8

Point is, taking the dot product of vectors over the field of complex numbers, \(\vec a ^\dagger \vec b\) (with \(\dagger\) is the Hermitian conjugate) really just turns your n-dimensional complex norm into the 2n-dimensional real norm.

Answer 9

So, say I give you the real vector
\[\vec v = \begin{pmatrix} a\\ b \\ c \\ d\end{pmatrix}\]
Then the norm (length squared) will just be:
\[\vec v^\top \vec v = a^2+b^2+c^2+d^2\]
But it might be nice to represent the vector this way
\[\vec v = \begin{pmatrix} a+bi \\ c+di \end{pmatrix}\] then you'd have to complex conjugate as well so that you can calculate its norm to be exactly the same as it was before
\[\vec v^\dagger \vec v = a^2+b^2+c^2+d^2\]
So it's sorta this bit that's important to have but, well it really depends on what you're wanting to do. Are you taking a linear algebra class or a quantum mechanics class?

Answer 10

I dropped out of school, but I'm still interested in mathematics and linear algebra in particular.
The reason I asked this question is because I'm trying to understand how the least squares problem generalizes to the complex case.

Answer 11

In the real case it would be (b-Ax) is a vector. That vector has to be perpendicular to the column space of A in order to make Ax as close as possible to b. so we would end up with A^T(b-Ax) = 0.
This is valid because this would set the dot product of the individual column vectors of A with the vector (b-Ax) to be equal to 0, which is true if cos(angle) = 0, thus the angle is 90 degrees, a.k.a. perpendicular.

Answer 12

Oh cool, I don't know much about least squares. Off the top of my head, I at least know that it's supposed to find the line that minimizes the distance between a bunch of points. To me, least squares is just a weird way of saying least distance. Since the difference between "least squares" and "least distance" is just a square root sign:these are

squares:\[x^2+y^2+z^2\]distance:\[\sqrt{x^2+y^2+z^2}\]
if you minimize one, you minimize the other. It's just easier to minimize without the square root there. You can actually show that with calculus,

\[f(x)=\sqrt{g(x)}\]
Now we know that \(f(x)\) is minimized when the derivative is 0. Let's take the derivative (use the chain rule):
\[f'(x)=\frac{g'(x)}{2\sqrt{g(x)}}\]
Now the min of f is when f'(x)=0 so plug that in to get:
\[0=\frac{g'(x)}{2\sqrt{g(x)}}\]
We know \(g(x)\ne 0\) otherwise we'd be dividing by 0, so it won't matter, and we can multiply both sides by \(2 \sqrt{g(x)}\) go get:
\[0=g'(x)\]
Hey, that means \(f'(x)=0\) happens exactly when \(g'(x)=0\)! So we can use least squares instead of least distance for minimization... Uhh sorry for that tangent.

Answer 13

thus: all the solutions would be given by row reducing A^T A x = A^T b.
then finding 1 solution for this problem and then adding the Null Space of (A^T A)^T.

Answer 14

Wait hold up so A is the position vectors of all the points and v is the vector you want to make as the line of least squares or what's going on here? Or can you just point me to a website you're reading and I'll refresh my memory on this.

Answer 15

but in the complex case this doesn't hold up.
in the complex case it would be A^H A x = A^H b.
where ^H means hermitian transpose.
but I don't understand why this is, because to me it seems like the only condition that has to be true is that the angle between the vectors is 90 degrees and thus I end up with Re(A^H A x) = Re(A^H b)

Answer 16

Yeah that's true, that's what it will have to be since we're going to end up with extra imaginary terms now.

Answer 17

https://www.khanacademy.org/math/linear-algebra/alternate-bases/orthogonal-projections/v/lin-alg-a-projection-onto-a-subspace-is-a-linear-transforma

Answer 18

unfortunately this video only covers the real case

Answer 19

Maybe the imaginary part ends up measuring something else about the quality of the least squares, like something related to the standard deviation or something.

Answer 20

Yeah that's what I thought as well. Now the problem is, I don't know if it measures anything about the quality of the least squares and if it does what it is, so that's why I came to openstudy to ask for help :P

Answer 21

Basically what it comes down to is, I don't understand why it is A^H A x = A^H b instead of Re(A^H A x = A^H b) in the complex case and I'm trying to get an intuitive understanding of this difference.

Answer 22

Well, now that you know, I think you can use that comment I made earlier along with your proof posted earlier and step through it and at least see what the expression is. The cross product (the imaginary part) has the interpretation of being the area of a parallelogram made from two vectors:

Answer 23

Wait, it's not "instead of" it's extra!

" A^H A x = A^H b instead of Re(A^H A x = A^H b)"

See, since
A^H A x = A^H b necessarily implies Re(A^H A x = A^H b)

Answer 24

\[\Im<a,b> = \sum_{k=0}^{n}(\Im(a_k)\Re(b_k) - \Re(a_k)\Im(b_k))\]

Answer 25

yes that's correct, but it doesn't imply the other way around

Answer 26

so the extra bit is Im(<a,b>)

Answer 27

and what I'm trying to understand is how this extra bit relates to the least squares quality

Answer 28

Yeah, I am working out stuff on paper to try to see what I can figure out right now.

Answer 29

Ah ok thank you :)

Answer 30

Oh here's an idea, is the imaginary part 0 if Ax=b ?

Answer 31

Hmmm that might be true indeed. I don't know how that helps though.

Answer 32

Is there anything I could clarify more to explain what I don't understand or is my problem clear to you?

Answer 33

Well what sort of help are you looking for? I think at this point there's nothing really left to help with, I feel like this is just sorta asking "what's the extra information give us?" Well we could throw it away and be fine I think.

Answer 34

so would solving Re(A^H A x = A^H b) be enough?

Answer 35

or does it have to be A^H A x = A^H b?

Answer 36

I think you gotta bite the bullet on that and just work out a small example or try to prove it yourself

Answer 37

Well I can think of an example where solving Re(A^H A x) = Re(A^H b) isn't enough... e.g. A = 1 and b = i

Answer 38

I just don't understand how this can be since when this condition is met, the angle is 90 degrees and thus it implies that there is no x that brings Ax closer to be :S

Answer 39

closer to b i mean.

Answer 40

pick x such that Re(1 * 1 x) = Re(1 * i) gives x = 0 + Re(L) * i
where L is any number.

Answer 41

obviously this can't be right since x just has to be equal to i in order to bring A x as close to b as possible and nothing else, thus Re(A^H A x) = Re(A^H b) isn't enough to solve this instead it has to be Re(A^H A x) = Re(A^H b) AND Im(A^H A x) = Im(A^H b)... but I don't understand why this is.

Answer 42

since Re(A^H (b - Ax)) has to be equal to 0 to make (b-Ax) perpendicular to the row space of A^H which implies that it IS enough :S so there must be something I am missing here or I've discovered some really weird paradox.

Answer 43

Does this make sense to you?

Answer 44

It's pretty late for me, so I'll come back to this tomorrow and see what I can do. Too complicated for my brain tonight lol.

Answer 45

haha ok no problem :P
thanks for helping me.