Wald Wolfowitz Runs Test

A single variable indepence test.

In a nutshell

The Wald-Wolfowitz Runs Test checks the randomness of a sequence of elements where those elements can take one of two mutually exclusive values. It is a non-parametric test.

Applications

Test the randomness of a distribution: \ Given a sequence of values from a distribution, each value can be labelled as being above the median '+', below the median '-', or omitted if equal to the median. While keeping the labels in the same order in which their corresponding values were sampled, the labels can be used as input to the Runs Test to check for randomness of the original distribution.

Sources: NIST/SEMATECH e-Handbook of Statistical Methods

\ Test for the independence of two groups: \ Taking one sample each from two groups. If the values can be ordered in some meaningful way (e.g. lowest to highest), we can then check whether these groups represent different populations. This is done by labelling each value according to the group from which it was taken, and then ordering the data in some meaningful way. The labels, once ordered according to their attached values, are then inspected for 'runs'.

Sources: Complete Dissertation

\ Do specific data and a function fit? \ Label the data that exceeds the function value with $+$ and the other data with $-$. Take as input to the function some independent variable about the data.

Problems:

• the Runs Test takes into account signs but not distance. This can be accounted for by complementing it with the Chi-Squared test which takes into account distances but not signs.
• the source information suggests labelling data that equals the function value with $-$, but this seems non-sensible seeing this data would support the hypothesis that the data and function fit.
  • my imagined solution is that equal data should not effect the number of runs but still be registered in the count of data points for hypothesis testing.
    • it is possible there is sensibility in the original scheme, despite my misgivings.

Sources: Wikipedia | NIST/SEMATECH e-Handbook of Statistical Methods

Example

Question: Based on a single variable that we've measured, does group 1 come from a different population to group 2?

Two groups of people, X and Y. For each group we have taken measurements of height.

Test steps:

1. Collect the data
2. Combine measurements from each group, but label each measurement to keep track of which group it came from.
3. Sort the data.

4. Along the sorted data, count the 'runs' of consecutive measurements with the same group label.

   For example, the following sequence has 5 such runs:

$$\begin{matrix} sequence\ \ \ \ \ \ : & y, & x, & x, & x, & y, & y, & x, & y \\ run\ number: & 1 & 2 & - & - & 3 & - & 4 & 5 \end{matrix} $$

1. Using a reference distribution as reference, compare the number of runs $R$ to the lower and upper critical values corresponding to the desired level of signifincance $\alpha$.

   • For 'large' samples ($n_1>10$ and $n_2>10$) compare to standard normal

   • Otherwise, use tables available elsewhere (Mendenhall, 1982)

     In the following example we'll use a 'small' sample size but still use a normal distribution as reference for simplicity's sake.

import numpy as np
from numpy.random import PCG64, default_rng
import array_to_latex as a2l
import os
import math
import string

out_dir = "wald-wolfowitz-outputs"

1. Collect the data

rng = default_rng(PCG64(2))
n_A = n_B = 7

# simulation height data from https://ourworldindata.org/human-height
# parameters
# # females
mu_1 = 164.7
sigma_1 = 7.07
# # males
mu_2 = 178.4
sigma_2 = 7.59

# Two samples: one sample from females, & one samples from males
A = [round(y, 1) for y in rng.normal(mu_1, sigma_1, n_A)]
B = [round(x, 1) for x in rng.normal(mu_2, sigma_2, n_B)]

$$A=\{\begin{matrix}166.0 & 161.0 & 161.8 & 147.4 & 177.4 & 172.8 & 162.4 \\ \end{matrix}\} $$$$B=\{\begin{matrix}184.3 & 180.5 & 174.2 & 185.8 & 176.0 & 175.9 & 172.4 \\ \end{matrix}\} $$

2 Label

# label each of our 'participants' with the group they are a part of
labelled_A = list(zip(A, ['A' for i in range(n_A)]))
labelled_B = list(zip(B, ['B' for i in range(n_B)]))

$$\begin{matrix}A & A & A & A & A & A & A \\ 166.0 & 161.0 & 161.8 & 147.4 & 177.4 & 172.8 & 162.4 \\ \end{matrix} $$$$\begin{matrix}B & B & B & B & B & B & B \\ 184.3 & 180.5 & 174.2 & 185.8 & 176.0 & 175.9 & 172.4 \\ \end{matrix} $$

3 Combine and Sort

labelled_sorted_all = sorted(labelled_A + labelled_B)

$$\begin{matrix}A & A & A & A & A & B & A & B & B & B & A & B & B & B \\ 147.4 & 161.0 & 161.8 & 162.4 & 166.0 & 172.4 & 172.8 & 174.2 & 175.9 & 176.0 & 177.4 & 180.5 & 184.3 & 185.8 \\ \end{matrix} $$

4. Count

# extract the ordered labels
ordered_labels = list(list(zip(*labelled_sorted_all))[1])

# set a counter
n_runs = 0
# set a variable to remember what the previous label was
previous_label = ""

for label in ordered_labels:
    if(label == ""):
        print("error: invalid label")
        break
    elif(previous_label != label):
        n_runs += 1
    previous_label = label

print("Numner of runs r is: " + str(n_runs))

Numner of runs r is: 6

5. Test

n = len(ordered_labels)

mu_0 = (n_A * n_B)/(n_A + n_B) + 1
sigma_0 = ((mu_0 - 1) * (mu_0 - 2))/(n-1)
lower = mu_0 - 1.96*sigma_0
upper = mu_0 + 1.96*sigma_0

reject = not (lower <= n_runs and n_runs <= upper)


if(reject):
    print("Reject the null hypothesis")
else:
    print("Do not reject null")

Reject the null hypothesis

Addendum: Runs Test algorithm where $F(x) = G(x)$

Let's do the same thing but with samples taken from the same population

# functions defined below

from ipynb.fs.full.wald_tests_generalized import sim

rng = default_rng(PCG64(2))
n_i = 7
is_same_pop = True
sim(n_i, is_same_pop, rng)

Number of data points: 	 14
Number of runs r is: 7
Expected Number of runs is 4.333333333333333
Do not reject null

...and from different populations again just to check we get the same results as from the start of the notebook

rng = default_rng(PCG64(2))
n_i = 7
is_same_pop = False
sim(n_i, is_same_pop, rng)

Number of data points: 	 14
Number of runs r is: 6
Expected Number of runs is 4.333333333333333
Reject the null hypothesis