---
title: "The optimal statistic and its null tests"
description: "Turn pair correlations into one amplitude estimate, fit several patterns at once, and calibrate the result with phase shifts and sky scrambles."
canonical_url: "https://sandboxphysics.com/stories/pulsar-timing-arrays/optimal-statistic-and-null-tests/"
language: "en"
page_type: "story"
alternate_languages:
  zh-CN: "https://sandboxphysics.com/zh/stories/pulsar-timing-arrays/optimal-statistic-and-null-tests/"
last_modified: "2026-11-14"
---

[Pulsar timing arrays and the Hellings–Downs curve](https://sandboxphysics.com/stories/pulsar-timing-arrays/)

# The optimal statistic and its null tests

Sandbox Physics · 2026-11-14

The optimal statistic turns the residuals of many pulsars into one estimate of the amplitude of a Hellings–Downs-correlated signal. Each pair of pulsars gives a noisy cross-correlation; the statistic is the weighted least-squares fit of the Hellings–Downs curve to all of them. Its formal signal-to-noise ratio is not a number of standard deviations, so the significance is calibrated empirically, by destroying the correlations in the data and counting how often noise alone does as well. Fitting a monopole and a dipole at the same time keeps clock and ephemeris errors from being counted as gravitational waves. This chapter builds the statistic, fits several patterns at once, and shows what the calibrations do, using the real NANOGrav pulsar positions.

You need the previous chapter, [deriving the Hellings–Downs curve](https://sandboxphysics.com/stories/pulsar-timing-arrays/deriving-the-hellings-downs-curve/), matrices, and weighted least squares.

## One number per pair

Let $\delta t_a$ be the residuals of pulsar $a$, $P_a$ their total auto-covariance, and $\tilde\Phi_{ab}$ the cross-covariance that a correlated signal of unit amplitude and unit correlation would produce. NANOGrav writes the pair estimate as

$$
\rho_{ab}=\frac{\delta t_a^{\mathsf T}P_a^{-1}\tilde\Phi_{ab}P_b^{-1}\delta t_b}{\operatorname{Tr}\big(P_a^{-1}\tilde\Phi_{ab}P_b^{-1}\tilde\Phi_{ba}\big)},\qquad \sigma_{ab}^2=\Big[\operatorname{Tr}\big(P_a^{-1}\tilde\Phi_{ab}P_b^{-1}\tilde\Phi_{ba}\big)\Big]^{-1}.
$$

Its expectation is $A^2\Gamma(\zeta_{ab})$. The variance $\sigma_{ab}^2$ holds when there is no correlated signal; a signal adds terms of order $A^4$. [NANOGrav Collaboration, 2023, Section 4, eq. 9](https://arxiv.org/abs/2306.16213v1)

Each $\rho_{ab}$ is far too noisy on its own. The array gains by having many pairs: 67 pulsars make 2211.

## The amplitude estimate

Fitting $\rho_{ab}\approx\hat A^2\,\Gamma(\zeta_{ab})$ by weighted least squares gives

$$
\hat A^2=\frac{\sum_{a>b}\rho_{ab}\Gamma_{ab}/\sigma_{ab}^2}{\sum_{a>b}\Gamma_{ab}^2/\sigma_{ab}^2},\qquad \frac{S}{N}=\frac{\sum_{a>b}\rho_{ab}\Gamma_{ab}/\sigma_{ab}^2}{\big(\sum_{a>b}\Gamma_{ab}^2/\sigma_{ab}^2\big)^{1/2}}.
$$

[NANOGrav Collaboration, 2023, eqs. 10 and 11](https://arxiv.org/abs/2306.16213v1) The S/N is $\hat A^2$ divided by its spread when there is no signal.

Worked example: how many pairs it takes

Suppose every pair had the same $\sigma$. Then $S/N=(A^2/\sigma)\sqrt{\sum\Gamma_{ab}^2}$. For the 67 NANOGrav pulsar positions the mean of $\Gamma_{ab}^2$ over the 2211 pairs is $0.0304$, so $\sqrt{\sum\Gamma^2}=\sqrt{67.2}=8.2$. An S/N of 4 needs $A^2/\sigma\approx0.5$ for a typical pair: each pair alone sits at half its own noise, and only the sum stands out.

## Why the S/N is not a sigma

Under the null hypothesis $\hat A^2$ is only approximately normal, so its S/N does not map onto a probability analytically. NANOGrav says it cannot be read as a sigma level. [NANOGrav Collaboration, 2023, Section 4](https://arxiv.org/abs/2306.16213v1) The p-value is built from data in which correlations have been removed on purpose.

A phase shift adds an independent random phase to every Fourier component of every pulsar. Each pulsar keeps its own spectrum, but the phases no longer line up between pulsars, so any inter-pulsar correlation is destroyed. A sky scramble keeps the data and moves the pulsars to random positions, which destroys the link between correlation and angle while keeping any correlation that does not depend on angle. NANOGrav ran 400 000 phase shifts; 19 had a noise-marginalised S/N above the observed one, giving $p=5\times10^{-5}$. Simulations and an analytic calculation gave $p=1.8\times10^{-4}$ and $1.9\times10^{-4}$.

Our own offline simulation shows why the calibration matters. It places synthetic Fourier coefficients at the real pulsar positions with white noise only, and computes the single-shape S/N 2000 times. The S/N exceeded 3 in $1.0\,\%$ of trials, while a normal distribution would give $0.13\,\%$.

## Fitting three shapes at once

A clock error gives every pair the same correlation, $\rho_{ab}=A_{\mathrm{clk}}^2$. Put that into the single-shape estimate and

$$
\hat A^2=A_{\mathrm{clk}}^2\,\frac{\sum_{a>b}\Gamma_{ab}/\sigma_{ab}^2}{\sum_{a>b}\Gamma_{ab}^2/\sigma_{ab}^2}.
$$

For pulsars spread evenly over the sky the numerator would average to zero, because the curve has no monopole. The real array is not even. With equal weights, the NANOGrav positions give a mean $\Gamma_{ab}$ of $0.047$ and a ratio of $1.54$: a clock error enters the background estimate at one and a half times its own size. The same ratio for an ephemeris dipole is $0.97$.

The multiple-component optimal statistic fits $\rho_{ab}\approx A^2_{\mathrm{HD}}\Gamma_{ab}+A^2_{\mathrm{mono}}+A^2_{\mathrm{dip}}\cos\zeta_{ab}$ in one weighted least-squares problem, so each shape is estimated with the others held accountable. [NANOGrav Collaboration, 2023, Section 5.3](https://arxiv.org/abs/2306.16213v1) Tiburzi and colleagues showed with simulated arrays that clock and ephemeris errors can produce false detections, and that removing ephemeris errors is not straightforward. [Tiburzi et al., 2015, Abstract and Section 4](https://arxiv.org/abs/1510.02363v1)

**Table 4.1** Offline simulation at the 67 NANOGrav positions, 300 trials per row: fraction of trials with S/N above 3.

| Injected signal | Single shape, background | Three shapes, background | Three shapes, clock | Three shapes, ephemeris |
| --- | --- | --- | --- | --- |
| background | 0.33 | 0.28 | 0.05 | 0.03 |
| clock error | 0.41 | 0.06 | 0.72 | 0.03 |
| ephemeris error | 0.31 | 0.02 | 0.09 | 0.80 |

The single-shape fit flags a clock error as a background in 41 % of trials. The three-shape fit cuts that to 6 % and moves the signal into the clock column. It also costs something: with a real background injected, a monopole above 3 still appears in 5 % of trials.

## Error bars under a real signal

Once a signal is present, pairs that share a pulsar are correlated with each other, and the scatter of $\hat A^2$ grows beyond the null value. In our simulation with a background injected, the scatter of the estimate was $2.2$ times the null uncertainty, and the null $1\sigma$ interval contained the injected value in only $31\,\%$ of trials. Allen adds that some of this scatter is cosmic variance, which no number of pulsars removes. [Allen, 2022, Sections IV and V](https://arxiv.org/abs/2205.05637v3)

NANOGrav's binned correlation plot accounts for these covariances. Neglecting them makes the error bars 20 to 40 % smaller. Its 15-bin reconstruction gives $\chi^2=8.1$ against the Hellings–Downs curve, a p-value of $0.75$. [NANOGrav Collaboration, 2023, Section 4](https://arxiv.org/abs/2306.16213v1)

## Lab task

Load the clock-error preset in the pulsar-timing-array Lab. The bar chart fits the three shapes at once; check that the signal lands in the clock bar while the background bar stays near zero. Then lower the array to eight pulsars and press New draw a few times to see how much the bars jump.

## Exercises

1. With equal $\sigma$ for every pair and $A^2/\sigma=0.3$, what S/N would the 2211 NANOGrav pairs give?
2. Check the phase-shift p-value: what is $(19+1)/(400\,000+1)$?
3. Why does a phase shift leave the noise spectrum of each pulsar unchanged? Why does a sky scramble not remove a monopole?
4. If the true scatter of $\hat A^2$ is $2.2$ times the null uncertainty, what fraction of null $1\sigma$ intervals should contain the true value, for a normal distribution?
5. A clock error with $A^2_{\mathrm{clk}}=1\times10^{-30}$ is fitted with the Hellings–Downs shape alone on the NANOGrav positions with equal weights. What amplitude squared comes out?

Answers

1. $0.3\times8.2=2.5$.
2. $5.0\times10^{-5}$, as printed.
3. The power in each Fourier component depends only on the size of its coefficient, which a phase shift does not change; only the alignment between pulsars is lost. A monopole gives every pair the same correlation, and moving the pulsars does not change that.
4. The interval covers when $|z|<1/2.2=0.45$, which has probability $0.35$. The simulation found $0.31$.
5. $1.54\times10^{-30}$, larger than the clock error itself.

## [Try it in the pulsar-timing-array Lab](https://sandboxphysics.com/labs/pulsar-timing-array-correlations/)

Load the clock-error preset and check that the fit puts the signal in the clock bar, not the background bar.

## Sources

1. [NANOGrav Collaboration, 2023, Section 4, eq. 9](https://arxiv.org/abs/2306.16213v1)
2. [Tiburzi et al., 2015, Abstract and Section 4](https://arxiv.org/abs/1510.02363v1)
3. [Allen, 2022, Sections IV and V](https://arxiv.org/abs/2205.05637v3)

## Keep reading

Pulsar timing arrays and the Hellings–Downs curve · Reading path

### From one ticking pulsar to a Galaxy-sized detector

1. [Article **How can a pulsar timing array tell gravitational waves from a bad clock?**](https://sandboxphysics.com/stories/pulsar-timing-arrays/gravitational-waves-or-a-bad-clock/)
2. [Article **What a pulsar timing residual measures**](https://sandboxphysics.com/stories/pulsar-timing-arrays/what-a-timing-residual-measures/)
3. [Article **Deriving the Hellings–Downs curve**](https://sandboxphysics.com/stories/pulsar-timing-arrays/deriving-the-hellings-downs-curve/)
4. [You are here **The optimal statistic and its null tests**](https://sandboxphysics.com/stories/pulsar-timing-arrays/optimal-statistic-and-null-tests/)
5. [Article **Recompute a Hellings–Downs statistic from NANOGrav 15-year data**](https://sandboxphysics.com/stories/pulsar-timing-arrays/recompute-the-optimal-statistic/)
6. [Article **Reading the NANOGrav 15-year evidence for a gravitational-wave background**](https://sandboxphysics.com/stories/pulsar-timing-arrays/reading-the-nanograv-15-year-evidence/)
