Question

Statistics

Answer 1

i believe that i did part A of this question correctly, as i was able to manipulate the equation to incorporate Xbar, but as for the other parts, i'm having difficulty understanding the actual process that needs to be done

Answer 2

I'm not too sure about the "completeness" question. My textbook never mentioned anything about complete statistics.

For part (c), you want to construct a function of \(\bar{X}\) such that the expected value of this function, let's call it \(\phi(X)\), is \(\beta\).

To make sure that \(\bar{X}\) is indeed sufficient for \(\beta\), here's the likelihood function:
\[L(\beta)=\prod_{i=1}^n f_{X_i}(x)=\prod_{i=1}^n\frac{1}{\beta}\exp\left(-\frac{x_i}{\beta}\right)=\frac{1}{\beta^n}\exp\left(-\frac{1}{\beta}\sum_{i=1}^n x_i\right)\]
By the factorization theorem, you can see that \(\sum X_i\) is sufficient for \(\beta\). \(\bar{X}\) is a function of this suff. stat., so it is also sufficient.

Now, the expected value of \(\bar{X}\):
\[E(\bar{X})=E\left(\frac{1}{n}\sum_{i=1}^nX_i\right)=\frac{1}{n}\sum_{i=1}^nE(X_i)=\beta\]
Thus, \(\bar{X}\) by itself is unbiased.

Answer 3

Finding the variance is fairly easy:
\[V(\bar{X})=V\left(\frac{1}{n}\sum X_i\right)=\frac{1}{n^2}\sum V(X_i)=\frac{\beta^2}{n}\]

Answer 4

An MVUE is any estimator that is unbiased and a function of a sufficient statistic. \(\bar{X}\) is both of these, and so it is an MVUE.

Answer 5

My professor defined a Complete Sufficient Statistic as follows:
Suppose that\[U=\left\{ U _{1},...,U _{n} \right\}\]is a sufficient statistic for estimation of theta, then we say that U is also a "complete sufficient" if\[E \left[ g(U)| \theta \right]=0\]for all theta implies that\[P \left[ g(U)=0|\theta \right]=1\]
Or in other words, U is complete sufficient for theta if the only choice of a function g, such that\[E \left[ g(U)|\theta \right]=0\]is the zero function (in probability) given by\[P \left[ g(U)=0|\theta \right]=1\]

Answer 6

I don't know if you can make sense of that, as i know i'm having trouble with it, which is why i asked this question.

Answer 7

To me, this somewhat resembles the def for consistent statistic... I don't think that's the same though.

Answer 8

A final question actually, what's the difference between using\[f(x|\beta)=\beta \exp(-\beta x)\]and using\[f(x|\lambda)=\frac{ 1 }{ \lambda }\exp(-\frac{ x }{ \lambda })\]should i start using the second one that you used in your explanation instead of the first one?

Answer 9

If you show in a) that your function belongs to the regular exponential family, then the natural sufficient statistic is a complete sufficient statistic, which is easier than using the definition of a complete statistic.

Answer 10

^ @SithsAndGiggles used this paramerization for the exponential pdf since they say the mean is \(\beta\). So you want \(E(X)=\beta\). That why the parameter was used as \(1/\beta\) rather than \(\beta\).

Recall that if the pdf is parameterized as follows:\[ f(x)=\lambda e^{-\lambda x}\], then the mean is \(E(X)=1/\lambda\). So if you use \(1/\beta\) instead of \(\lambda\), then the mean is \(\large E(X)=\frac{1}{1/\beta}=\beta\)

Answer 11

ahh, okay, thank you for that clarification, that's something i need to pay a lot more attention to then

Answer 12

Not sure if this helps but in the first paragraph here, http://www.public.iastate.edu/~vardeman/stat543/Handouts/Lehmann-Scheffe.pdf, they describe the conditions for a complete statistic

Answer 13

I don't know whether I'm being dense or just not following what the natural sufficient statistict that is being referred to is, sorry I've been jumping around the internet looking for multiple clarifications on everything, my professor never mentioned anything on exponential family yet, only my section TA has mentioned them, briefly.

Answer 14

hm. Well, if you can write the pdf of a distribution in the following form:
\[f(x)=c(\theta)h(x)\exp\left[\sum_{j=1}^k \eta_j(\theta) T_j(x)\right]\], then the distribution belongs to the exponential family.

Let's look at a more "complicated" example to illustrate this, because I think it is a bit trivial with the exponential distribution.

Let's take a normal distribution \(N(\mu, \sigma^2)\), and transform it into the exponential family form above. Note that \(\theta\) is/are your parameter(s). So it can be a vector. Here, \(\theta=(\mu, \sigma^2)\).
\(c(\theta)\) can only be a function of \(\theta\), \(h(x)\) can only be a function of \(x\), and \(\eta_j(\theta)\) are functions of \(\theta\) only but must be inside the \(\exp\) function and as a product with \(T_j(x)\), which are functions of \(x\).

\[f(x)=\frac{1}{\sqrt{2\pi}\sigma}\exp \left[ -\frac{1}{2\sigma^2}(x-\mu)^2\right] \\
=\frac{1}{\sqrt{2\pi}\sigma}\exp\left[ -\frac{1}{2\sigma^2}\left( x^2-2\mu x+\mu^2\right)\right]\\
=\frac{1}{\sqrt{2\pi}\sigma}\exp\left[ -\frac{1}{2\sigma^2} x^2+\frac{1}{\sigma^2}\mu x\right]\exp\left[-\frac{1}{2\sigma^2}\mu^2\right]\]

\[c(\theta)=\frac{1}{\sqrt{2\pi}\sigma}\exp\left[-\frac{1}{2\sigma^2}\mu^2\right]\]\[ h(x)=1\]\[ \eta_1(\theta)=-\frac{1}{2\sigma^2}, \,\,T_1(x)=x^2\]\[ \eta_2(\theta)=\frac{\mu}{\sigma^2},\,\, T_2(x)=x\]
So, from this, you establish that \(\left(\sum_{i=1}^nT_1(X_i),\ldots,\sum_{i=1}^nT_k(X_i)\right)\) is the natural sufficient statistic, if you have a random sample of size \(n\).

In the example above, you would say that \(\left(\sum_{i=1}^nX_i,\sum_{i=1}^nX_i^2\right)\) is your natural sufficient statistic.

Answer 15

The notation you have seen for the exponential family though might be a bit different though from what I used.

Answer 16

Now, there's a theorem that states that:
if you show that
1) your distribution can take that exponential form,
2) neither the \(T_j\) no \(\eta_j\) satisfy any linear constraints, and
3) the parameter space contains a \(k\)-dimensional rectangle (i.e. it contains an open subset in \(\mathbb{R}^k\))
then, the natural sufficient statistic is a complete sufficient statistic.

Answer 17

Hm actually showing completeness explicitly by the definition with the exponential distribution is not too complicated here. I can show you below (once you finish typing)

Answer 18

i was just going say how much sense your explanation made, but please, go ahead :D

Answer 19

it fixed some notation errors that were in my notes that were causing some very bad mistakes

Answer 20

Oh fantastic :)
Ya usually showing completeness is easiest by the method above, since it works for any distribution belonging to the exponential family. Using the definition is usually more complicated, but it is fairly simple for the exponential distribution:

From your notation \(U=\bar{X}\) is your sufficient statistic. Now, \(\bar{X}\) is a function of \(X\). So, Let's find \(E[g(U)]\):
\[ E[g(U)]=\int_{0}^{\infty}g(u)\cdot\frac{1}{\beta}e^{-\frac{1}{\beta}u}du\]
So, \(E[g(U)]=0\) if and only if:
\[ E[g(U)]=\int_{0}^{\infty}g(u)\cdot\frac{1}{\beta}e^{-\frac{1}{\beta}u}du=0\\
\implies\frac{1}{\beta}\int_0^{\infty}g(u)e^{-\frac{1}{\beta}u}dx=0\]
Now, you can never have \(1/\beta=0\), and an exponential can never be 0, so the only part that makes the above statement true is if \(g(u)=0\). So, \(g(u)=0\) for all \(\theta\). In other words, this ALWAYS occurs (with probability 1).

Thus, \(E[g(U)]=0\) for all \(\theta\) implies \(P[g(U)=0]=1\) for all \(\theta\).

Answer 21

And just to make sure that i'm completely on board with this,\[E \left[ g(U) \right]=\int\limits_{0}^{\infty}g(u)\frac{ 1 }{ \beta }e ^{\frac{ -u }{ \beta }}du\]comes from the fact that expectation is calculated as\[E \left[ X \right]=\int\limits_{0}^{\infty}xf \left( x \right)dx\]where in my case, since i have a exponential distribution, i have that\[f \left( x \right)=\frac{ 1 }{ \beta }e ^{\frac{ -x }{ \beta }}\]

Answer 22

indeed !

Answer 23

Oh my gosh, thank you so much, you just helped me understand what i've been having a serious problem with for the past week, and it only took a few posts, i can't even begin to thank you enough!

Answer 24

Awesome :)!!