1.3 Other common distributions

Here, we briefly present some of the distributions that we will need or use in this book. Throughout, we assume independent and identically distributed data.

1.3.1 The standard normal: \(\mathit{normal}(\mu=0,\sigma=1)\)

Although we have already seen this distribution, the standard normal distribution is so important for us that it deserves its own section.

The standard normal is a special case of the normal distribution and is written \(\mathit{normal}(0,1)\).

One important fact that is relevant for us in this book is that if \(X_1, \dots, X_n\) are independent and identically distributed random variables from a distribution with mean \(\mu\) and variance \(\sigma^2\), then, as \(n\) approaches infinity, the distribution of the transformed random variable \(Z\) is:

\[\begin{equation} Z = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \end{equation}\]

\(\bar{X}\) is the mean of the random variables \(X_1,\dots, X_n\): \(\bar{X}=\sum_{i=1}^n \frac{X_i}{n}\).

The pdf of the standard normal is just like the \(\mathit{normal}(\mu,\sigma)\) we saw above, with \(\mu = 0\) and \(\sigma = 1\).

\[\begin{equation} f(z)= \frac{1}{\sqrt{2\pi}} \exp \left(-\frac{z^2}{2} \right) \end{equation}\]

1.3.2 The uniform distribution

A continuous random variable \(U\) has a uniform distribution if the pdf of \(U\) is:

\[\begin{equation} f(u; \alpha, \beta)= \begin{cases} \frac{1}{\beta-\alpha} & for \alpha < u < \beta ,\\ 0 & \hbox{otherwise} \end{cases} \end{equation}\]

The mean and variance of the \(\mathit{uniform}(\alpha,\beta)\) are:

\[\begin{equation} \mu = \frac{\alpha + \beta}{2} \hbox{ and } \sigma^2 = \frac{1}{12} (\beta-\alpha)^2 \end{equation}\]

Figure 1.7 shows a \(\mathit{uniform}(\alpha=0,\beta=1)\) distribution.

u<-seq(0,1,by=0.001)
plot(u,dunif(u,min=0,max=1),type="l",
     ylab="density",main="uniform(0,1)")

FIGURE 1.7: The pdf of the uniform(0,1) distribution.

The cdf \(F(U<u)\) has the property that \(F(U<u)=u\):

punif(0.25,min=0,max=1)

## [1] 0.25

punif(0.75,min=0,max=1)

## [1] 0.75

Figure 1.8 shows the cdf for the distribution uniform(0,1).

FIGURE 1.8: The cumulative distribution function of the uniform distribution uniform(0,1).

1.3.3 The Chi-square distribution

A random variable \(X\) that has a chi-square distribution has the following pdf:

\[\begin{equation} f(u; \nu)= \begin{cases} \frac{1}{2^{\nu/2} \Gamma(\nu/2)}x^{\frac{\nu-2}{2}\exp(-\frac{x}{2})} & for x>0 ,\\ 0 & \hbox{otherwise} \end{cases} \end{equation}\]

The parameter \(\nu\) is referred to as the degrees of freedom. \(\Gamma()\) is the Gamma function: \(\Gamma(n)=(n-1)!\).

The mean and variance of \(\chi^2_{\nu}\) are:

\[\begin{equation} \mu = \nu \hbox{ and } \sigma^2 = \nu^2 \end{equation}\]

Figure 1.9 shows a \(\chi^2_{\nu=1}\) distribution.

x<-seq(0,4,by=0.001)
plot(x,dchisq(x,df=1),type="l",
     ylab="density",main="Chi-square(df=1)")

FIGURE 1.9: The pdf of the Chi-square distribution with 1 degree of freedom.

An important fact about the \(\chi^2_{\nu=1}\) distribution that we will use in chapter 4 is that the quantile \(x\) such that \(F(X>x)=0.0.05\) is \(x=3.84\).

qchisq(0.05,df=1,lower.tail=FALSE)

## [1] 3.841

1.3.4 The t-distribution

A random variable \(T\) that has a t-distribution has the pdf:

\[\begin{equation} f(t) = \frac{\Gamma(\frac{\nu+1}{2})}{\sqrt{\pi\nu}\Gamma(\frac{\nu}{2})}\left(1+\frac{t^2}{\nu}\right)^{-\frac{\nu + 1}{2}} \hbox{ for } -\infty < t < \infty \end{equation}\]

This distribution is also called the Student’s t-distribution; the creator of the t-distribution, Gosset, wrote papers under the name Student.

This continuous distribution, often written \(t(n-1)\), takes as a parameter the degrees of freedom \(\nu=n-1\), where \(n\) is the sample size (what constitutes sample size will be made more precise in the next chapter). As the degrees of freedom approaches infinity (as the sample size approaches infinity), the distribution approaches the Normal distribution with mean 0 and standard deviation 1.

The mean of the t-distribution is \(0\) if \(n>1\) and variance \(\frac{n}{n-2}\) if \(n>2\).

The t-distribution becomes the Cauchy distribution if \(n=2\). The Cauchy distribution has no mean or variance defined for it.

The t-distribution is actually related to the chi-square and standard normal distributions. If \(Y\) and \(Z\) are independent random variables and \(Y\) has a chi-square distribution with \(\nu\) degrees of freedom and \(Z\) has the standard normal, then the t-distribution with \(\nu\) has the following form:

\[\begin{equation} T = \frac{Z}{\sqrt{Y/\nu}} \end{equation}\]

There are four functions in R that serve the same purpose as the dnorm, pnorm, qnorm, rnorm functions for the Normal: dt, pt, qt, rt. These functions, particularly dt, pt, qt, will play a very important role in chapter 2.

1.3.5 The F distribution

If \(U\) and \(V\) are independent random variables with a chi-square distribution having degrees of freedom \(\nu_1\) and \(\nu_2\) respectively, then the following pdf is called an F-distribution:

\[\begin{equation} F = \frac{U/\nu_1}{V/\nu_2} \end{equation}\]

The pdf of the F-distribution is:

\[\begin{equation} g(f) = \frac{\Gamma(\frac{\nu_1 + \nu_2}{2})}{\Gamma(\frac{\nu_1}{2}) \Gamma(\frac{\nu_2}{2}) } \left(\frac{\nu_1}{\nu_2}\right)^{\nu_1/2} f^{\nu_1/2 -1 } (1 + \frac{\nu_1}{\nu_2}f)^{-\frac{1}{2}(\nu_1 + \nu_2)} \end{equation}\]

for \(f>0\). The pdf \(g(f)\) has value \(0\) for all other values of \(f\).