2.3 Three examples of the sampling distribution

In the above discussion, the underlying pdf we sampled from above was a normal distribution. However, it need not be. Consider two examples first: the underlying pdf is an Exponential or a Gamma distribution.

The Exponential distribution has a parameter \(\lambda\) (parameterized in R as a rate, \(1/\lambda\)); its mean is \(\lambda\) and its variance is \(1/\lambda^2\). The sampling distribution is normal, even though the underlying distribution is an Exponential; see Figure 2.3.

FIGURE 2.3: Sampling from an exponential.

A further example is samples from a Gamma distribution. The Gamma distribution has two parameters, a and b, and is written Gamma(a,b). In R, the parameters a and b are called shape and rate, respectively. The mean of the Gamma distribution is \(\frac{a}{b}\) and the variance is \(\frac{a}{b^2}\). Suppose we sample from a Gamma distribution with shape parameter chosen arbitrarily to be 1. The distribution of means is again going to be approximately normal; see Figure 2.4.

FIGURE 2.4: Sampling from a Gamma distribution.

As a final example, consider what happens if we sample from a distribution, the Cauchy, that doesn’t have any mean or variance defined for it.

FIGURE 2.5: Sampling from a Cauchy distribution.

As Figure 2.5 illustrates, when the mean and variance for the likelihood are undefined, the central limit theorem doesn’t hold. In the rest of this book, we will always assume that the data are coming from a distribution that has a mean and variance defined for it.