The standard Normal distribution

The standard Normal distribution#

The standard normal distribution is perhaps the most recognizable continuous distribution. It has that characteristic bell shape. We write:

Z \sim N (0, 1),

and we read it as:

$Z$ follows the standard Normal.

Now the 0 in $N (0, 1)$ is the expected value and the 1 is the variance. We will see that later.

The PDF of the standard Normal#

We commonly use the function $ϕ (z)$ to represent the PDF of the standard Normal. It is:

ϕ (z) = \frac{1}{\sqrt{2 π}} \exp {- \frac{z^{2}}{2}} .

So, this is an exponential that has $z^{2}$ inside it. The term $\frac{1}{\sqrt{2 π}}$ is there so that the PDF is normalized, i.e.:

\int_{- \infty}^{+ \infty} ϕ (z) d z = 1.

Here is how you can make a standard Normal in scipy.stats:

Z = st.norm()

And here are some samples from it:

Z.rvs(size=10)

array([ 0.07650491, -0.55433992,  0.35401783,  0.26572118, -0.5496704 ,
        0.47381155, -0.53989222,  1.13163748,  1.23093609,  0.42745344])

And here is the PDF of the standard normal:

fig, ax = plt.subplots()
zs = np.linspace(-6.0, 6.0, 100)
ax.plot(zs, Z.pdf(zs))
ax.set_xlabel('$z$')
ax.set_ylabel('$\phi(z)$');

../_images/8d9e070faf73099fa5f872bcd85f93632f5785e47765c2d2b3538380edcea836.svg

Here are some important properties of the PDF of the standard Normal:

First, $ϕ (z)$ is positive for all $z$ .
Second, as $z$ goes to $- \infty$ or $+ \infty$ , $ϕ (z)$ goes to zero.
Third, $ϕ (z)$ has a unique mode (maximum) at $z = 0$ . In other words, $z = 0$ is the most probable point under this distribution.
Fourth, $ϕ (z)$ is symmetric about $z = 0$ . Mathematically:

ϕ (- z) = ϕ (z) .

Let’s test some of them using scipy.stats.

Z.pdf(np.inf)

0.0

Z.pdf(-np.inf)

0.0

Z.pdf(-5) == Z.pdf(5)

True

Expectation of the standard normal#

The expectation of $Z$ is:

E [Z] = \int_{- \infty}^{+ \infty} z ϕ (z) d z = 0.

You can prove this quite easily by invoking the fact that $ϕ (- z) = ϕ (z)$ .

Here it is in scipy.stats:

Z.expect()

0.0

Variance of the standard normal#

The variance of $Z$ is:

V [Z] = \int_{- \infty}^{+ \infty} z^{2} ϕ (z) d z = 1.

You need integration by parts to prove this. Let’s do it.

\begin{array}{r} \begin{aligned} V [Z] & = \int_{- \infty}^{+ \infty} z^{2} ϕ (z) d z \\ = \frac{1}{\sqrt{2 π}} \int_{- \infty}^{+ \infty} z^{2} \exp {- \frac{z^{2}}{2}} d z \\ = \frac{1}{\sqrt{2 π}} \int_{- \infty}^{+ \infty} (- z) \cdot [- z \exp {- \frac{z^{2}}{2}}] d z \\ = - \frac{1}{\sqrt{2 π}} \int_{- \infty}^{+ \infty} z \cdot \frac{d}{d z} [\exp {- \frac{z^{2}}{2}}] d z \\ = - \frac{1}{\sqrt{2 π}} {{[z \exp {- \frac{z^{2}}{2}}]}_{- \infty}^{+ \infty} - \int_{- \infty}^{+ \infty} \exp {- \frac{z^{2}}{2}} d z} \\ = - \frac{1}{\sqrt{2 π}} (0 - \sqrt{2 π}) \\ = 1. \end{aligned} \end{array}

Note that the standard deviation of $Z$ is also 1 ( $Z$ does not have units).

Again, here it is in scipy.stats:

Z.var()

1.0

The CDF of the standard Normal#

The CDF of $Z$ gives you the probability that $Z$ is smaller than a number $z$ . It is common to use $Φ (z)$ to denote the CDF. There is no closed form. But you can write this:

Φ (z) = \int_{- \infty}^{z} ϕ (z^{'}) d z^{'},

where $ϕ (z)$ is the PDF of $Z$ . Let’s plot it.

fig, ax = plt.subplots()
ax.plot(zs, Z.cdf(zs))
ax.set_xlabel('$z$')
ax.set_ylabel('$\Phi(z)$');

../_images/bf3a36108b8a1515088d4f05604b1ede491e7f9338b0a3a77a887a1e839b08ac.svg

Some properties of the CDF of the standard normal#

First, note that:

Φ (0) = 0.5 .

This follows very easily from the symmetry of the PDF $ϕ (z)$ about zero. Remember that $Φ (0)$ is the probability that $Z$ is smaller than zero and because of symmetry that probability is exactly 50%. A point $z$ with such a property (the probability that the random variable is smaller than it is 0.5) is called the median of the random variable.

Now here is a non-trivial property. Take any number $z$ . Then, we have:

Φ (- z) = 1 - Φ (z) .

Before attempting to prove this property, let’s demonstrate it visually. Take $z$ to be some positive number. Then $Φ (- z)$ is the probability that $Z$ is smaller than $- z$ , or this area below the PDF:

Φ (- z) = \int_{- \infty}^{- z} ϕ (z^{'}) d z^{'} .

On other hand, $1 - Φ (z)$ is the following area:

\begin{array}{r} \begin{aligned} 1 - Φ (z) & = \int_{- \infty}^{+ \infty} ϕ (z^{'}) d z^{'} - \int_{- \infty}^{z} ϕ (z^{'}) d z^{'} \\ = \int_{- \infty}^{z} ϕ (z^{'}) d z^{'} + \int_{z}^{+ \infty} ϕ (z^{'}) d z^{'} - \int_{- \infty}^{z} ϕ (z^{'}) d z^{'} \\ = \int_{z}^{+ \infty} ϕ (z^{'}) d z^{'} . \end{aligned} \end{array}

Notice that we used the fact that $ϕ (z)$ is normalized and a standard property of the integral.

Alright, so visually, the expression $Φ (- z) = 1 - Φ (z)$ means that the red and the blue areas in the following plot are the same for any $z$ :

../_images/1a030c46a7b38e252b7c1e927fbf5486d74901ed95c6fd15c597a07d5f60246b.svg

And this makes a lot of sense since $ϕ (z)$ is symmetric. So, in words the property says:

The probability that $Z$ is smaller than $- z$ is the same as the probability that $Z$ is greater than $z$ .

The formal proof is actually trivial. It goes like this:

\begin{array}{r} \begin{aligned} Φ (- z) & = \int_{- \infty}^{- z} ϕ (z^{'}) d z^{'} \\ = \int_{+ \infty}^{z} ϕ (\tilde{z}) (- 1) d \tilde{z}, \end{aligned} \end{array}

after applying the transformation $\tilde{z} = - z^{'}$ . And finally:

\int_{+ \infty}^{z} ϕ (\tilde{z}) (- 1) d \tilde{z} = - \int_{+ \infty}^{z} ϕ (\tilde{z}) d \tilde{z} = \int_{z}^{+ \infty} ϕ (\tilde{z}) d \tilde{z},

which as we saw above is the same as $1 - Φ (z)$ .

Let’s demonstrate this in scipy.stats:

print('p(Z <= -1) = {0:1.3f}'.format(Z.cdf(-1)))

p(Z <= -1) = 0.16

And this should be the same as the probability that $Z$ is greater than 1, which is:

print('p(Z >= 1) = {0:1.3f}'.format(1 - Z.cdf(1)))

p(Z >= 1) = 0.16

What is the probability that $Z$ is between $- 1$ and $1$ ? It is:

p (- 1 < Z < 1) = 1 - p (Z \leq - 1) - p (Z \geq 1) = 1 - 2 Φ (- 1) .

In scipy.stats:

print('p(-1 < Z < 1) = {0:1.3f}'.format(1 - 2.0 * Z.cdf(-1)))

p(-1 < Z < 1) = 0.683

Questions#

Modify the code above to find the probability that $Z$ is between -2 and 2.
Repeat for $Z$ between -3 and 3.