Question

(Bayesian Estimation) Let \(X_i\sim\text{N}(\mu,\theta^{-1})\) be \(n\) iid random variables. \(\mu\) and \(\theta\) are unknown. The prior has a Normal-Gamma distribution with parameters \(\alpha_1,\alpha_2,\beta_1,\beta_2\) as follows: \[ \pi(\mu,\theta)=c\theta^{\beta_1/2}\exp\left\{-\frac{\theta}{2}\left[ \alpha_1+\beta_2(\alpha_2-\mu)^2\right] \right\}, c\text{ is a constant }, \theta>0, \mu \in \mathbb{R}\]Show that that the prior distribution is a conjugate prior.

Answer 1

Hmm I will try this. Give me a moment to try it and type it...

Answer 2

Denote \(L(\mu,\theta)\) as the likelihood, \(\pi(\mu, \theta)\) as the prior, and \(\pi(\mu,\theta|x)\) as the posterior. Also, I will use \[s^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar{x})^2\]
\[\begin{align*}L(\mu,\theta)&=\prod_{i=1}^n\frac{1}{\sqrt{2\pi}\sqrt{\frac{1}{\theta}}}\exp\left\{-\frac{1}{2\cdot\frac{1}{\theta}} (x_i-\mu)^2\right\}\\
&=(2\pi)^{-n/2}\left(\frac{1}{\theta}\right)^{-n/2}\exp\left\{-\frac{\theta}{2}\sum_{i=1}^n(x_i-u)^2 \right\} \end{align*}\]
\[\pi(\mu, \theta|x)\propto L(\mu,\theta)\pi(\mu,\theta)\\
=(2\pi)^{-n/2}\theta^{n/2}c\theta^{\beta_1/2}\exp\left\{-\frac{\theta}{2}\sum_{i=1}^n(x_i-u)^2-\frac{\theta}{2}\left[\alpha_1+\beta_2(\alpha_2-\mu)^2\right] \right\}\\
\propto\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\sum_{i=1}^n\left((x_i-\bar{x})+(\bar{x}-u)\right)^2-\frac{\theta}{2}\left[\alpha_1+\beta_2(\alpha_2^2-\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:2a_2\mu+\mu^2)\right] \right\}\\
=\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\left[ (n-1)s^2+n(\bar{x}-\mu)^2 \right]-\frac{\theta}{2}\left[\alpha_1+\beta_2\alpha_2^2-\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:2\beta_2a_2\mu+\beta_2\mu^2\right] \right\}\\
=\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\left[ (n-1)s^2+n\bar{x}^2-2n\bar{x}\mu+n\mu^2+\alpha_1+\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\beta_2\alpha_2^2-2\beta_2a_2\mu+\beta_2\mu^2\right] \right\}\\
=\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\left[ (n-1)s^2+\alpha_1+\beta_2\alpha_2^2+n\bar{x}^2+\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:(n+\beta_2)\mu^2-2\mu(n\bar{x}+\beta_2\alpha_2)\right] \right\}\\
=\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\left[ (n-1)s^2+\alpha_1+\beta_2\alpha_2^2+n\bar{x}^2+\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:(n+\beta_2)\left(\mu^2-\frac{2\mu(n\bar{x}+\beta_2\alpha_2)}{n+\beta_2}\right)\right] \right\}\\
=\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\left[ (n-1)s^2+\alpha_1+\beta_2\alpha_2^2+n\bar{x}^2+\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:(n+\beta_2)\left(\mu^2-\frac{2\mu(n\bar{x}+\beta_2\alpha_2)}{n+\beta_2}+\left(\frac{n\bar{x}+\beta_2\alpha_2}{n+\beta_2}\right)^2-\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\left(\frac{n\bar{x}+\beta_2\alpha_2}{n+\beta_2}\right)^2\right)\right] \right\}\\
=\theta^{(n+\beta_1)/2}\exp\left\{-\frac{\theta}{2}\left[ (n-1)s^2+\alpha_1+\beta_2\alpha_2^2+n\bar{x}^2+\\
\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:(n+\beta_2)\left( \frac{n\bar{x}+\beta_2\alpha_2}{n+\beta_2}-\mu\right)^2-\frac{(n\bar{x}+\beta_2\alpha_2)^2}{n+\beta_2}\right] \right\}\]

Hence, you have a normal-gamma distribution with parameters:
\[\alpha1:=(n-1)s^2+\alpha_1+\beta_2\alpha_2^2+n\bar{x}^2-\frac{(n\bar{x}+\beta_2\alpha_2)^2}{n+\beta_2}\\
\alpha_2:=\frac{n\bar{x}+\beta_2\alpha_2}{n+\beta_2}\\
\beta_1:=n+\beta_1\\
\beta_2:=n+\beta_2\]
Since the posterior is normal-gamma distributed like the prior, then the prior is a conjugate prior

Answer 3

I hope this is correct...

Answer 4

Wow amazing!! Looks right to me. Thank you sooo much!