Question

Show that \(f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R} \) with \(f(x,y)= \| x-y \|_2^2 \) is differentiable and compute it's differential at every point in the domain of \(f\).
Note that \(\|\cdot\|_2\) denotes the euclidian norm here.

Answer 1

I am aware of the definition, not on how to find the differential though. When an assistant Professor told me to compute the directional derivate \[\large \lim_{\epsilon \to 0} \frac{f(x+ \epsilon h_1, y + \epsilon h_2) -f(x,y)}{\epsilon}\]
which can be done for the above function in every direction with result \( 2 <x-y,h_1-h_2> \) however I cannot see how the existence of directional derivatives (although in all directions) implies total differentiability here. There are a few pathological examples in which this is not the case, even worse, not even continuity occurs.

Professor @eliassaab

Answer 2

@experimentX, maybe you have a thought or two on this one :)

Answer 3

I don't understand what it means by \( 2<x-y, h_1-h_2 > \)
however, the function is differentiate at each point \((x,y) \in \Bbb R^n \times \Bbb R^n \). To do that, you need to show that the limit is same from all directions.
\[ \large \lim_{\epsilon \to 0} \frac{f(x+ \epsilon h_1, y + \epsilon h_2) -f(x,y)}{\epsilon}\]

Answer 4

this is simplification of ,
\[ \lim_{(\Delta x,\Delta y) \to (0, 0)} \frac{f(x+\Delta x, y+\Delta y) - f(x,y)}{\sqrt{\Delta x^2 + \Delta y^2}}\]
which is the definition of derivative of two variable functions. Just to show that the derivative exists, you compute that limit.

Answer 5

The ease that comes with computing
\[ \large \lim_{\epsilon \to 0} \frac{f(x+ \epsilon h_1, y + \epsilon h_2) -f(x,y)}{\epsilon} \]
is that \( (h_1, h_2) \) represents some direction. and you show that this limit is independent of direction. It's not that difficult.

Given that \( f(x,y) \) is Euclidean metric on RxR, the value is
\[ \lim_{\epsilon \to 0} \frac{\sqrt{(x + \epsilon h_1)^2 +(y + \epsilon h_2)^2 } - \sqrt{x^2 +y ^2}}{\epsilon } \]

multiply it by conjugate and find the limit. Show that the limit is independent of \((h_1, h_2)\)

Answer 6

Thanks, I understand that part and have computed it before, although I might want to do it again using the notion of the dot product. In my problem they are talking about the euclidian norm squared, hence it all breaks down to the notion of \(<x,y>\) or sometimes denoted as \(x \cdot y\) in physics, where \((x,y)\) is just a positive definite, bilinear, symmetric form (scalar product)

However, as I have mentioned above, there are many pathological examples out there for which the directional derivative exists in all direction but the function itself is not differentiable. I was wondering about an approach with using the total differential.

Then again I believe that your method is right, (if and only if though) we include that the derivative has to be a linear operator

Answer 7

Here are also some of the calculations \[f(x+ \epsilon h_1, y + \epsilon h_2)= \sum((x-y)+ \epsilon (h_1-h_2))^2 \\ = \sum(x-y)^2+ 2 \sum((x-y) \epsilon (h_1-h_2))+ \sum( \epsilon (h_1-h_2))^2 \\
\text{s.t } f(x+ \epsilon h_1, y + \epsilon h_2)- f(x,y)= 2 \sum((x-y) \epsilon (h_1-h_2)) + \sum(\epsilon(h_1 -h_2))^2 \]
Division by \(\epsilon\) and taking the limit as \(\epsilon \to 0\) would lead to \[2 \sum(x-y)(h_1-h_2) = 2<x-y,h_1-h_2> \]

Answer 8

@Zarkon

Answer 9

@amistre64, maybe you have taken this course (-: !

Answer 10

looks like I took the wrong map,
you really get
$$2 \langle x-y, h_1 - h_2 \rangle$$

Answer 11

I think i smell some problem here ... according to definition of derivative from my analysis book, derivative is defined at a point, i.e.
A function f is differentiable at a 'c' if , there exists a linear function \(T_c(v):\Bbb R^n \to \Bbb R^m \) such
$$f(c+v) = f(c) + T_c(v) + ||v|| E_c(v)$$
where \( E_c(v) \to 0 \) as \( v\to 0\).

The problem here is we have two points on space \( x, y \in \Bbb R^3\), now where to define differentiability?

Answer 12

I think that above definition is for \( (x,y) \in \Bbb R^2 \), or rather we need, \((x, y) \in \Bbb R^6 \)

Answer 13

I agree @experimentX, I find this rather difficult myself. However I've seen some proofs of the statement which in general use the approach you suggested, I just yet fail to see how this implies differentiability ( or rather total differentiability). Maybe @eliassaab can have a word or two on the definition :)

Answer 14

try asking here ... http://math.stackexchange.com/questions
let's see what people here have to say.

Answer 15

also don't forget to send me the link ... i'll give you +1

Answer 16

http://math.stackexchange.com/questions/767282/show-that-fx-y-x-y-22-is-differentiable/767295#767295 I have asked that question there already two days ago :-) I was mainly 'reposting' it on here, because I understand one answer (the one given by Pedro Tamaroff) but I am puzzled about the approach of Professor Siminore which is in fact equivalent to yours.

It seems to be a theorem that I am missing, so I did post it again on here because I thought it might be something I overlook at only needs a nudge.

Answer 17

Is that bilinear??

Answer 18

http://math.stackexchange.com/a/767307/2199
according to his definition, ... i think what you proved is more than enough.

Answer 19

just the middle term is bilinear. \[(x-y) \cdot (x-y) = x \cdot x - 2 x \cdot y + y \cdot y = \|x\|^2 - 2 x \cdot y + \| y\|^2 \]
so his approach is to show that the middle bilinear term is always differentiable, hence we have the sum of differentiable functions.

Answer 20

what is the definition of differentiable from your book?

Answer 21

yeh I believe so, it's just the first time I see it using that way, because the very first thing I have learnt about directional derivatives is to be very careful with them, as they very rarely imply total differentiability.

I study mathematics, so it's the same as the one you posted above, or the one I have given in the post. \[\text{f is differentiable at } x_0 \iff \exists A \in \hom( \mathbb{R}^m , \mathbb{R}^n) : \lim_{x \to x_0} \frac{f(x)-f(x_0)-A(x-x_0)}{\|x-x_0\|} =0\]

Answer 22

\[\text{f is diff' at } x_0 \iff \exists A \in \hom( \mathbb{R}^m , \mathbb{R}^n) : \lim_{x \to x_0} \frac{f(x)-f(x_0)-A(x-x_0)}{\|x-x_0\|} =0 \]

Answer 23

what's hom ?? homomorphism?

Answer 24

Exactly, a linear mapping.

Answer 25

\(A\) denotes a matrix for instance, the Jacobi-Matrix, or any linear mapping.

Answer 26

homomorphism is linear??

Answer 27

i thought homomorphism is just
f(ab) = f(a)f(b)

Answer 28

yes that is homomorphisms are defined. http://en.wikibooks.org/wiki/Linear_Algebra/Definition_of_Homomorphism (in linear algebra)

there are homomorphism on group structures that are defined differently.

Answer 29

well well ... this is new!! what level are you studying btw?

Answer 30

I am BSc in Mathematics, 2nd year. This is Linear Algebra II/III, for whatever that means. It differs from county to country and I am european. Our courses here are called a bit differently, like Calculus is "Real Analysis" here.

Answer 31

Mathematics is Europe is sure tough ... I studied physics for major. I studied Apostle's analysis during my undergrad, but ... I never encountered something like it.

Answer 32

I am at a "Elite University" (we don't have that term here, officially - the universes are in Europe prohibited from calling themselves that) but it's the ETH (Polytechnikum), as a physicist you may have heard about it. It is where Einstein studied and was a Professor at.

Answer 33

*in Europe

Answer 34

It's basically like any University worldwide, just that they love giving challenging problems and then call us blunt for not being able to solve them (x

Answer 35

I don't know the name ... but it's in Germany Munich somewhere.

Answer 36

woops!! this one??
http://en.wikipedia.org/wiki/ETH_Zurich

Answer 37

Switzerland, where the LHC is located at, the Large Hadron Collider, well the Collider is in Geneve, ETH is in Zurich. Yes the link is correct.

Answer 38

Europe used to be center for maths ... especially Germany Gottengen.

Answer 39

Hope it gets better once my main courses are done with and I can specialize a bit more, I sure like the american education system with having minors and majors.

Answer 40

IDK about american system, but your maths is sure tough ... I went further to get BS in maths, but compared to that one, it was easy. Looking weather to go to grad school in maths or just should quit.

Answer 41

but never encountered anything as you posted above. I am Asian btw ..

Answer 42

anyway let's see what can we do with your definition or with answers for M.SE
accordig to siminore's answer, it's plainly obvious. Let's see ... if I can decrypt Peter's answer.

Answer 43

Pedro's answer I understand, mainly just an application of the bilinear module and then some fancy definition for a mapping.

Siminore's answer sure is correct, I believe as a Professor of Mathematics in Italy he knows what he is doing, but it makes no sense to me.

Googling for the existence of directional derivatives gives immediate answers about that this is not enough. However, he mentions 'linearity in the increment' which somehow seems to fix things.

Answer 44

probably you mean this comment
```
Yes, of course. The directional derivative may exists but it may not be linear in the
increment. In this case the function can't be differentiable (since the derivative must be a
liner operator).
```

Answer 45

well well ... it doesn't make sense to me either :(

Answer 46

what's this linear in increment ...

Answer 47

hehe that's no problem, in fact it's a relief that not only I don't understand that. It's not big of a deal, I did copy that approach anyway and will ask my instructor when he returns about that comment, I am sure there is some nifty subtle nuance to it.

I believe he considers \(h_1,h_2\) as 'increments', although I would have called them directions, which is really what they are.

The result shows that it can be written as a scalar/dot product, hence it is bilinear and therefore linear in the first and in the second entries.

Answer 48

My next attempt was to take the result \(2 <x-y,h_1-h_2>\) and plug it into the definition to see if it holds, but I wasn't able to compute it. By definition I mean the 'rigorous' definition as given in my and your textbook.

Answer 49

you mean this thing?
\[ \text{f is diff' at } x_0 \iff \exists A \in \hom( \mathbb{R}^m , \mathbb{R}^n) : \lim_{x \to x_0} \frac{f(x)-f(x_0)-A(x-x_0)}{\|x-x_0\|} =0 \]

Answer 50

still it is defined for a single variable ... what Pedro claims is ... if we are able to show that each of the term in x^2 - 2xy + y^2 is differentiable then it's differentiable.

Answer 51

exactly, tried to substitute it for the \(A\) and verify if I have found the correct differential.

Answer 52

all right ... we know x^2 and y^2 is differenable ... and xy is bilinear, and if I recall there exists a matrix for binlinear mapping
xBz such that \( B : \Bbb R^n \times \Bbb R^m \to \Bbb R \)

Answer 53

The elegant thing pedro does is mainly evaluating the Bilinear form and then showing that the first part can be described as a linear mapping \(A\) and the second part is a quantity that approaches \(0\) as \((h,k) \to (0,0)\)

the \(|h_i| |k_j| \leq \|h,k\| \) part I don't quite get but it makes intuitive sens

Answer 54

no no ,,, there is comma between them.

Answer 55

oh yup, I saw that, my typesetting was flawed (-:

Answer 56

good thing about Pedro's answer is .. it is consistent with the definition of derivative.
I would choose his answer over Siminore's +1 for him.

Answer 57

he shows that
\[ f(x+h,y+k) = f(x , y) + A(h,k) + C ||h, k||\]

Answer 58

I meant why \(|h_i|, |k_j| \leq ||h,k||\) the intuitive thing would have been for me to say \[ \large \| B(h,k) \| \leq \sum_{i,j} |h_i||k_j| \|B(e_i,e_j)\| \]
and then for \(|h_i| |k_j|\) choosing that for instance the one or the other has to be bigger, if \(|h_i| \geq |k_j|\) then I would have said that \[ |h_i| |k_j| \leq |h_i|2 \leq |h_1|^2 + \dots + |h_i|^2 + \dots + |h_n|^2=|h|^2 \]

Answer 59

yes I agree, his answer is definitely more of a proof while Siminore's is more a mechanical approach to it, a computation rather.

Answer 60

i forgot the name of this inequality ... let me refer to my Linear algebra book.

Answer 61

can't find it ... of course i gave it my sister.

Answer 62

okay ... just need to prove
\[ |h_i| \le \left| \sum_{i, j}^{n, m} h_i k_j\right|\]

Answer 63

Ugh!!
http://en.wikipedia.org/wiki/Bessel's_inequality

Answer 64

Thank you very much I will look into it and thanks a lot for all your help, I truly appreciate it. I unfortunately have to go now, if I later get some insight into the equations (from my instructor) or from Siminore's post I will forward it to here :) Thanks

Answer 65

sure ... i'll update if I find something interesting!!

Answer 66

\[ f(x+ h, y + h)- f(x,y)= 2 \sum((x-y) (h-k)) + \sum (h -k)^2 \\
= A(h_1, h_2) + ||h, k|| E(h, k) \]
Where \( E(h, k) = ||h, k|| \to 0\) as \( ||h, k|| \to 0 \) ... which seems to be consistent with definition of total derivative which also seems what that professor from Italy was trying to tell.