Skip to main content
Sandbox Physics
Pulsar timing arrays and the Hellings–Downs curve

The optimal statistic and its null tests

Sandbox Physics ·

The optimal statistic turns the residuals of many pulsars into one estimate of the amplitude of a Hellings–Downs-correlated signal. Each pair of pulsars gives a noisy cross-correlation; the statistic is the weighted least-squares fit of the Hellings–Downs curve to all of them. Its formal signal-to-noise ratio is not a number of standard deviations, so the significance is calibrated empirically, by destroying the correlations in the data and counting how often noise alone does as well. Fitting a monopole and a dipole at the same time keeps clock and ephemeris errors from being counted as gravitational waves. This chapter builds the statistic, fits several patterns at once, and shows what the calibrations do, using the real NANOGrav pulsar positions.

You need the previous chapter, deriving the Hellings–Downs curve, matrices, and weighted least squares.

One number per pair

Let δta\delta t_a be the residuals of pulsar aa, PaP_a their total auto-covariance, and Φ~ab\tilde\Phi_{ab} the cross-covariance that a correlated signal of unit amplitude and unit correlation would produce. NANOGrav writes the pair estimate as

ρab=δtaTPa−1Φ~abPb−1δtbTr⁡(Pa−1Φ~abPb−1Φ~ba),σab2=[Tr⁡(Pa−1Φ~abPb−1Φ~ba)]−1.\rho_{ab}=\frac{\delta t_a^{\mathsf T}P_a^{-1}\tilde\Phi_{ab}P_b^{-1}\delta t_b}{\operatorname{Tr}\big(P_a^{-1}\tilde\Phi_{ab}P_b^{-1}\tilde\Phi_{ba}\big)},\qquad \sigma_{ab}^2=\Big[\operatorname{Tr}\big(P_a^{-1}\tilde\Phi_{ab}P_b^{-1}\tilde\Phi_{ba}\big)\Big]^{-1}.

Its expectation is A2Γ(ζab)A^2\Gamma(\zeta_{ab}). The variance σab2\sigma_{ab}^2 holds when there is no correlated signal; a signal adds terms of order A4A^4. NANOGrav Collaboration, 2023, Section 4, eq. 9

Each ρab\rho_{ab} is far too noisy on its own. The array gains by having many pairs: 67 pulsars make 2211.

The amplitude estimate

Fitting ρab≈A^2 Γ(ζab)\rho_{ab}\approx\hat A^2\,\Gamma(\zeta_{ab}) by weighted least squares gives

A^2=∑a>bρabΓab/σab2∑a>bΓab2/σab2,SN=∑a>bρabΓab/σab2(∑a>bΓab2/σab2)1/2.\hat A^2=\frac{\sum_{a>b}\rho_{ab}\Gamma_{ab}/\sigma_{ab}^2}{\sum_{a>b}\Gamma_{ab}^2/\sigma_{ab}^2},\qquad \frac{S}{N}=\frac{\sum_{a>b}\rho_{ab}\Gamma_{ab}/\sigma_{ab}^2}{\big(\sum_{a>b}\Gamma_{ab}^2/\sigma_{ab}^2\big)^{1/2}}.

NANOGrav Collaboration, 2023, eqs. 10 and 11 The S/N is A^2\hat A^2 divided by its spread when there is no signal.

Worked example: how many pairs it takes

Suppose every pair had the same σ\sigma. Then S/N=(A2/σ)∑Γab2S/N=(A^2/\sigma)\sqrt{\sum\Gamma_{ab}^2}. For the 67 NANOGrav pulsar positions the mean of Γab2\Gamma_{ab}^2 over the 2211 pairs is 0.03040.0304, so ∑Γ2=67.2=8.2\sqrt{\sum\Gamma^2}=\sqrt{67.2}=8.2. An S/N of 4 needs A2/σ≈0.5A^2/\sigma\approx0.5 for a typical pair: each pair alone sits at half its own noise, and only the sum stands out.

Why the S/N is not a sigma

Under the null hypothesis A^2\hat A^2 is only approximately normal, so its S/N does not map onto a probability analytically. NANOGrav says it cannot be read as a sigma level. NANOGrav Collaboration, 2023, Section 4 The p-value is built from data in which correlations have been removed on purpose.

A phase shift adds an independent random phase to every Fourier component of every pulsar. Each pulsar keeps its own spectrum, but the phases no longer line up between pulsars, so any inter-pulsar correlation is destroyed. A sky scramble keeps the data and moves the pulsars to random positions, which destroys the link between correlation and angle while keeping any correlation that does not depend on angle. NANOGrav ran 400 000 phase shifts; 19 had a noise-marginalised S/N above the observed one, giving p=5×10−5p=5\times10^{-5}. Simulations and an analytic calculation gave p=1.8×10−4p=1.8\times10^{-4} and 1.9×10−41.9\times10^{-4}.

Our own offline simulation shows why the calibration matters. It places synthetic Fourier coefficients at the real pulsar positions with white noise only, and computes the single-shape S/N 2000 times. The S/N exceeded 3 in 1.0 %1.0\,\% of trials, while a normal distribution would give 0.13 %0.13\,\%.

Fitting three shapes at once

A clock error gives every pair the same correlation, ρab=Aclk2\rho_{ab}=A_{\mathrm{clk}}^2. Put that into the single-shape estimate and

A^2=Aclk2 ∑a>bΓab/σab2∑a>bΓab2/σab2.\hat A^2=A_{\mathrm{clk}}^2\,\frac{\sum_{a>b}\Gamma_{ab}/\sigma_{ab}^2}{\sum_{a>b}\Gamma_{ab}^2/\sigma_{ab}^2}.

For pulsars spread evenly over the sky the numerator would average to zero, because the curve has no monopole. The real array is not even. With equal weights, the NANOGrav positions give a mean Γab\Gamma_{ab} of 0.0470.047 and a ratio of 1.541.54: a clock error enters the background estimate at one and a half times its own size. The same ratio for an ephemeris dipole is 0.970.97.

The multiple-component optimal statistic fits ρab≈AHD2Γab+Amono2+Adip2cos⁡ζab\rho_{ab}\approx A^2_{\mathrm{HD}}\Gamma_{ab}+A^2_{\mathrm{mono}}+A^2_{\mathrm{dip}}\cos\zeta_{ab} in one weighted least-squares problem, so each shape is estimated with the others held accountable. NANOGrav Collaboration, 2023, Section 5.3 Tiburzi and colleagues showed with simulated arrays that clock and ephemeris errors can produce false detections, and that removing ephemeris errors is not straightforward. Tiburzi et al., 2015, Abstract and Section 4

Table 4.1 Offline simulation at the 67 NANOGrav positions, 300 trials per row: fraction of trials with S/N above 3.
Injected signalSingle shape, backgroundThree shapes, backgroundThree shapes, clockThree shapes, ephemeris
background0.330.280.050.03
clock error0.410.060.720.03
ephemeris error0.310.020.090.80

The single-shape fit flags a clock error as a background in 41 % of trials. The three-shape fit cuts that to 6 % and moves the signal into the clock column. It also costs something: with a real background injected, a monopole above 3 still appears in 5 % of trials.

Error bars under a real signal

Once a signal is present, pairs that share a pulsar are correlated with each other, and the scatter of A^2\hat A^2 grows beyond the null value. In our simulation with a background injected, the scatter of the estimate was 2.22.2 times the null uncertainty, and the null 1σ1\sigma interval contained the injected value in only 31 %31\,\% of trials. Allen adds that some of this scatter is cosmic variance, which no number of pulsars removes. Allen, 2022, Sections IV and V

NANOGrav's binned correlation plot accounts for these covariances. Neglecting them makes the error bars 20 to 40 % smaller. Its 15-bin reconstruction gives χ2=8.1\chi^2=8.1 against the Hellings–Downs curve, a p-value of 0.750.75. NANOGrav Collaboration, 2023, Section 4

Lab task

Load the clock-error preset in the pulsar-timing-array Lab. The bar chart fits the three shapes at once; check that the signal lands in the clock bar while the background bar stays near zero. Then lower the array to eight pulsars and press New draw a few times to see how much the bars jump.

Exercises

  1. With equal σ\sigma for every pair and A2/σ=0.3A^2/\sigma=0.3, what S/N would the 2211 NANOGrav pairs give?
  2. Check the phase-shift p-value: what is (19+1)/(400 000+1)(19+1)/(400\,000+1)?
  3. Why does a phase shift leave the noise spectrum of each pulsar unchanged? Why does a sky scramble not remove a monopole?
  4. If the true scatter of A^2\hat A^2 is 2.22.2 times the null uncertainty, what fraction of null 1σ1\sigma intervals should contain the true value, for a normal distribution?
  5. A clock error with Aclk2=1×10−30A^2_{\mathrm{clk}}=1\times10^{-30} is fitted with the Hellings–Downs shape alone on the NANOGrav positions with equal weights. What amplitude squared comes out?
Answers
  1. 0.3×8.2=2.50.3\times8.2=2.5.
  2. 5.0×10−55.0\times10^{-5}, as printed.
  3. The power in each Fourier component depends only on the size of its coefficient, which a phase shift does not change; only the alignment between pulsars is lost. A monopole gives every pair the same correlation, and moving the pulsars does not change that.
  4. The interval covers when ∣z∣<1/2.2=0.45|z|<1/2.2=0.45, which has probability 0.350.35. The simulation found 0.310.31.
  5. 1.54×10−301.54\times10^{-30}, larger than the clock error itself.

Sources

  1. NANOGrav Collaboration, 2023, Section 4, eq. 9
  2. Tiburzi et al., 2015, Abstract and Section 4
  3. Allen, 2022, Sections IV and V

Keep reading

What this page links to directly

See this series on the content map →

Pulsar timing arrays and the Hellings–Downs curve · Reading path

From one ticking pulsar to a Galaxy-sized detector

  1. ArticleHow can a pulsar timing array tell gravitational waves from a bad clock?
  2. ArticleWhat a pulsar timing residual measures
  3. ArticleDeriving the Hellings–Downs curve
  4. You are hereThe optimal statistic and its null tests
  5. ArticleRecompute a Hellings–Downs statistic from NANOGrav 15-year data
  6. ArticleReading the NANOGrav 15-year evidence for a gravitational-wave background
Something broken, confusing, or missing on this page?Tell us
What kind of feedback?

Sent with this page’s address, its Lab ID, the site language and your device type. Please leave out personal details. Feedback is kept for up to 12 months.Privacy policy