Question

OLS

Answer 1

\[OLS = \frac{ \sum_{i=1}^{n} Y_{i} X_{i} }{ \sum_{i=1}^{n} X_{i}^{2}}\]

Answer 2

\[ROM = \frac{ \sum_{i=1}^{n} Y_{i}} { \sum_{i=1}^{n} X_{i}}\]

Answer 3

\[MOR = \frac{ 1 }{ n } \sum_{i=1}^{n} \frac{ Y_{i} }{ X_{i} }\]

Answer 4

Since the regression is of a variable on a constant, I guess this is equivalent to \[y_{i}=4 + \epsilon_{i}\] with 4 just an example of some constant. We could then assume the expectation of residuals is zero so we are left with \[y_{i}=4\]But where next?

Answer 5

@phi @TuringTest ?

Answer 6

@e.mccormick @Ashleyisakitty

Answer 7

@amistre64

Answer 8

@myininaya can you help?

Answer 9

@karatechopper ?

Answer 10

@hartnn @Abhisar ?

Answer 11

Can you just eliminate the \[X_{i}\] in the equations altogether? Or does the constant become the \[X_{i}\] perhaps?

Answer 12

If you remove the \[X_{i}\] though you won't have the mean, certainly in the ROM case you'll just have the sum of the constant? Do I need to take expectations?

Answer 13

for a constant variable,
X1 = X2 =X3 =X4 =...

so \(X_i \) will be a constant

Answer 14

Okay so you could just call it any number? say, 4?

Answer 15

Or perhaps my interpretation of that is wrong?

Answer 16

call it 'a' where a is a constant

Answer 17

Okay great

Answer 18

So for this question would I then have to find the mean using the formulas for OLS, ROM and MOR above, using this constant a as all X1, X2, X3... values?

Answer 19

\(\sum X_i \)
will be total number of 'X' values

like for

X: 3,3,3,3,3,3,3
\(\sum X_i = 7\)

Answer 20

Okay sure. So with a, \[\sum_{}^{}X_{i} = an\] ?

Answer 21

i think i read it wrong..
it always takes a constant value, means something like this :


X : 1 2 3 4 5 6 7
Y : 1 1 1 1 1 1 1

...
so, infact,
\(\sum Y_i = 7\)
take a=1 for simplicity

Answer 22

Okay that was my initial thinking. But then I ran into trouble when I tried to plug such a setup into the three formulas

Answer 23

I will give it another try now. Maybe breaking down the different summations and seeing what results I get will help

Answer 24

I'm thinking \[\sum_{}^{}Y_{i}X_{i}=nx\] does that look right @phi @hartnn ?

Answer 25

Very confused. If that was right OLS goes to 1/x, which can't be right...

Answer 26

Any ideas @phi?

Answer 27

Someone wrote this online re: the question.. "Here is a hint, If you are fitting a regression with only a constant then you are fitting a horizontal line to the data. Think about the criteria that the regression models are using to find the "best" fit and then work out what value (height of the horizontal line) will fit that criteria."

Answer 28

I was thinking, for Ordinary Least squares, we say
\[ y_i= \alpha+ \beta x_i \\ \alpha = \bar{y} - \beta \bar{x} \\
y_i= \bar{y} - \beta \bar{x} + \beta x_i
\]

Answer 29

if the x's are constant then for all \(x_i, x_i= \bar{x} \)
and so
\[ y_i= \bar{y} - \beta \bar{x} + \beta x_i \\ = \bar{y} - \beta \bar{x} +\beta \bar{x} \\= \bar{y} \]

Answer 30

Thank you @phi that's got to be a big leap forward. I hadn't thought of breaking away from the formulas above entirely. That said, given the regression is just that on a constant, wouldn't the initial setup have to be \[y_{i}=\alpha\]

Answer 31

regression "on a constant" (I think) means the input (independent variable) is the constant.

Answer 32

y = fcn(x)

Answer 33

**** "Here is a hint, If you are fitting a regression with only a constant then you are fitting a horizontal line to the data..." ***
that is saying that the "slope" of the line is zero in the regression formula
\[ y_i = \alpha + \beta x_i\]
i.e. \( \beta=0\)

Answer 34

at least for OLS
\[ \alpha= \bar{y} - \beta \bar{x} \]
and with \( \beta=0 \) we again get
\[ y_i = \alpha = \bar{y} \]

Answer 35

So surely \[y_{i}= \alpha\]
and following your logic
\[\alpha = \bar{y}\] which proves the regression is always equal to mean of variable?

Answer 36

Yep great!! Thank you!! No idea on approach for ROM or MOR though?

Answer 37

I would hope it is something similar, but I don't know about ROM or MOR

Answer 38

I would write down the expression for \( y_i\)
using the "ROM" model. Any idea what it is ?

Answer 39

Unfortunately not. All I can find online is the same formula given above, which I was given in class. The intuition of ROM is that it finds the average value of X and the average value of Y and draws a line from origin to that point to make best fit. So you're finding intercept of Xbar and Ybar and drawing from origin

Answer 40

I guess to find solution for the 3 estimators one needs to use those formuals above just can't see how to apply it

Answer 41

if it's of the form y= b + m x
then we need the definition for b and m (though m = 0 if we assume the x's are constant)

Answer 42

As a best alternative to a formal solution I could perhaps describe the fact that in each of these cases they're linear estimators with no slope since we're working with only y = b?

Answer 43

yes. and then show that b is \( \bar{y} \)

Answer 44

Would you be able to provide the intuition behind this step? I've always had a bit of trouble with it

Answer 45

I don't see how to explicitly use your formulas for Beta, because the equations are indeterminate. Not surprising because we are find the slope = \( \frac{\Delta y}{\Delta x} \)
and we are dividing by 0

Answer 46

Agreed. I'll probably go with that descriptive description of the scenario

Answer 47

"descriptive description" whoops

Answer 48

**Would you be able to provide the intuition behind this step? ***
I will have to think about it, and post something later. But your question is a bit broad.
The derivation of least-squares can be done using linear algebra and projection matrices, or using calculus to minimize the sum of squares error. Either way, you end up with those formulas

Answer 49

Okay thank you so much for your help!

Answer 50

For what it is worth, here is Khan using linear algebra
https://www.khanacademy.org/math/linear-algebra/alternate_bases/orthogonal_projections/v/linear-algebra-least-squares-examples

Answer 51

@satellite73

Answer 52

How about this for
\[ r = \frac{ \sum_{i=1}^{n} Y_{i}} { \sum_{i=1}^{n} X_{i}} \]
the model is
\[ y_i = r x_i \]
if x is constant = a then
\[ \sum_{i=1}^{n} X_{i} = \sum_{i=1}^{n} a= na\]
and
\[ r = \frac{ \sum_{i=1}^{n} Y_{i}}{a n} = \frac{\bar{y}}{a}\]
using that r in the model we gat
\[ y_i =\frac{\bar{y}}{a} x_i \]
but all x's = a so
\[ y_i =\frac{\bar{y}}{a} a =\bar{y} \]

Answer 53

now we need the MOR model

Answer 54

@phi that is fantastic. thank you so much!! The step un using the n in the denominator to make for the ybar I really wouldn't have seen!