Tweedie distributions

Tweedie's contributions included pioneering work with the Inverse Gaussian distribution.^[3][4] Arguably his major achievement rests with the definition of a family of exponential dispersion models characterized by closure under additive and reproductive convolution as well as under transformations of scale that are now known as the Tweedie exponential dispersion models.^[1][5] As a consequence of these properties the Tweedie exponential dispersion models are characterized by a power law relationship between the variance and the mean which leads them to become the foci of convergence for a central limit like effect that acts on a wide variety of random data.^[6] The range of application of the Tweedie distributions is wide and includes:

• Taylor's law,^[7][8]
• fluctuation scaling,^[9]
• 1/f noise,^[8]
• random matrix theory,^[8]
• hematogenous cancer metastasis,^[10]
• genomic structure and evolution,^[11][12]
• regional blood flow heterogeneity,^[13]
• multifractality.^[8]
• self-organized criticality^[14]

Tweedie's formula

Tweedie is credited for a formula first published in Robbins (1956),^[15] which offers "a simple empirical Bayes approach to correcting selection bias".^[16] Let ${\displaystyle \mu }$ be a latent variable we don't observe, but we know it has a certain prior distribution ${\displaystyle p(\mu )}$. Let ${\displaystyle x=\mu +\epsilon }$ be an observable, where ${\displaystyle \epsilon \sim N(0,\Sigma )}$ is a Gaussian noise variable (so ${\displaystyle p(x|\mu )=N(x|\mu ,\Sigma )}$) . Let ${\displaystyle \rho (x)=\int p(x|\mu )p(\mu ){\text{d}}\mu }$ be the probability density of ${\displaystyle x}$, then the posterior mean and variance of ${\displaystyle \mu }$ given the observed ${\displaystyle x}$ are:

$${\displaystyle E[\mu |x]=x+\Sigma {\frac {\nabla \rho (x)}{\rho (x)}};\quad Var[\mu |x]=E[\mu \mu ^{T}|x]-E[\mu |x]E[\mu |x]^{T}=\Sigma \left({\frac {\nabla ^{2}\rho (x)}{\rho (x)}}-{\frac {\nabla \rho (x)\nabla \rho (x)^{T}}{\rho (x)^{2}}}\right)\Sigma +\Sigma }$$

The posterior higher order moments of ${\displaystyle \mu }$ are also obtainable as algebraic expressions of ${\displaystyle \nabla \rho ,\rho ,\Sigma }$.

Proof for first part

Using ${\displaystyle \nabla N(x|\mu ,\Sigma )=N(x|\mu ,\Sigma )\nabla \log N(x|\mu ,\Sigma )=-\Sigma ^{-1}(x-\mu )N(x|\mu ,\Sigma )}$, we get

$${\displaystyle \Sigma \,{\frac {\nabla \rho (x)}{\rho (x)}}={\frac {\Sigma \int \nabla N(x|\mu ,\Sigma )\,p(\mu ){\text{d}}\,\mu }{\int N(x|\mu ,\Sigma )\,p(\mu )\,{\text{d}}\mu }}=-{\frac {\int (x-\mu )N(x|\mu ,\Sigma )\,p(\mu )\,{\text{d}}\mu }{\int N(x|\mu ,\Sigma )\,p(\mu )\,{\text{d}}\mu }}=-\int (x-\mu )p(\mu |x)\,{\text{d}}\mu =-x+E[\mu |x],}$$

where we have used Bayes' theorem to write

$${\displaystyle p(\mu |x)={\frac {p(x|\mu )p(\mu )}{p(x)}}={\frac {N(x|\mu ,\Sigma )\,p(\mu )}{\int N(x|\mu ,\Sigma )\,p(\mu )\,{\text{d}}\mu }}.}$$

Tweedie's formula is used in empirical Bayes method and diffusion models.^[17]