Skip to main content
Sandbox Physics
Baryon acoustic oscillations

Measure a correlation function from a DESI data patch

Sandbox Physics ·

By the end of this guide you will have measured the galaxy correlation function of 15,699 real DESI galaxies, estimated its covariance with a jackknife, tried a BAO fit, and seen four ways the measurement goes wrong. One Python script does it all from saved files in about a minute and a half. The honest result is a non-detection: this patch is far too small to show the BAO bump, and the guide explains why. The galaxies are real DESI Data Release 1 data. The fit template is an offline theory calculation, not data.

Read the chapter on counting pairs with randoms first. It derives the estimator and the jackknife used here.

Get the files

Download these into one empty folder:

The expected output was made with Python 3.9.6, NumPy 2.0.2 and SciPy 1.13.1. From the folder, run:

python3 -m venv bao-env
bao-env/bin/pip install -r requirements.txt
bao-env/bin/python -I measure_desi_patch_correlation.py --output my-result.json

The flag -I keeps Python from importing anything from the current folder. Add --skip-failures to run only the main measurement, which takes about 20 seconds.

What is in the data

The galaxies come from the DESI DR1 large-scale structure catalogue BGS_BRIGHT-21.5 in the South Galactic Cap: bright galaxies with an absolute magnitude cut, the same sample DESI used for its lowest-redshift BAO measurement. The patch keeps right ascension −20∘-20^\circ to 20∘20^\circ, declination −10∘-10^\circ to 10∘10^\circ and redshift 0.1≤z<0.40.1 \le z < 0.4. The randoms come from the first of DESI's random files for the same sample. Inside the patch that file holds 1,043,189 points. The download keeps a random subsample three times the galaxy count, 47,097 points, to stay small.

Each row holds ra_deg, dec_deg, z, weight and weight_fkp. The weight column combines DESI's corrections for completeness and imaging systematics. The FKP column balances the sample against its changing number density. DESI 2024 III says the first is meant for any calculation and the second is included in all two-point measurements, so the script uses their product.

The provenance file records the SHA-256 hashes of both DESI input files and both outputs, and every change made: columns dropped, rows cut to the patch, randoms subsampled with a fixed seed, values rounded. The data are released under CC BY 4.0. If you reuse them, cite the DESI DR1 paper and keep the acknowledgment in the README.

Step 1: redshifts to distances

The script converts each redshift to a comoving distance in DESI's fiducial flat cosmology, Ωm=0.31519\Omega_m = 0.31519, using

DC(z)=c100 km s−1 Mpc−1∫0zdz′Ωm(1+z′)3+1−Ωm.D_C(z) = \frac{c}{100\,\mathrm{km\,s^{-1}\,Mpc^{-1}}}\int_0^z \frac{dz'}{\sqrt{\Omega_m(1+z')^3 + 1 - \Omega_m}} .

The prefactor makes distances come out in h−1 Mpch^{-1}\,\mathrm{Mpc}, the unit DESI uses for separations. Radiation is left out, which changes distances below z=0.4z = 0.4 by less than 0.01 percent. Right ascension, declination and distance then give Cartesian positions.

Step 2: weighted pair counts

The script builds k-d trees with SciPy and counts weighted pairs in twenty bins of 10 h−1 Mpc10\,h^{-1}\,\mathrm{Mpc} out to 200 h−1 Mpc200\,h^{-1}\,\mathrm{Mpc}. It counts data with data, data with randoms and randoms with randoms, normalises each by its total weight of pairs, and forms the Landy-Szalay estimate.

For the jackknife it divides the patch into an 8 by 4 grid on the sky. Two of the 32 cells contain no randoms, because they fall outside the DESI footprint, so 30 regions are used. Leaving out an empty region would just repeat the full sample and shrink the error bars.

Step 3: read the correlation function

Open my-result.json at correlation. Each row gives ss, ξ\xi, its jackknife error, and the same in s2ξs^2\xi.

Table 5.1 The measured correlation function near the BAO scale, from the expected output. Units of ss are h−1 Mpch^{-1}\,\mathrm{Mpc}.
ssξ\xis2ξs^2\xiError in s2ξs^2\xi
551.8681.86846.746.71.61.6
45450.04460.044690.390.325.325.3
85850.00890.008964.064.045.445.4
1051050.00520.005256.956.974.674.6
1251250.00170.001726.526.5102.4102.4
145145−0.0002-0.0002−4.4-4.4126.6126.6

At small separations the signal is strong and the errors are tiny. Near 100 h−1 Mpc100\,h^{-1}\,\mathrm{Mpc} the error in s2ξs^2\xi is larger than the value itself. Any bump there is hidden.

Step 4: the BAO fit

The fit uses the CLASS template computed for DESI's fiducial cosmology at z=0.295z = 0.295, with the linear power spectrum multiplied by exp⁡(−k2Σ2/2)\exp(-k^2\Sigma^2/2). Here Σ=7.84 h−1 Mpc\Sigma = 7.84\,h^{-1}\,\mathrm{Mpc} is the root mean square of DESI's pre-reconstruction BGS damping scales, 6.56.5, 6.56.5 and 10.0 h−1 Mpc10.0\,h^{-1}\,\mathrm{Mpc}. DESI's own model damps only the oscillating part of the spectrum; this simpler template damps all of it. In s2ξs^2\xi its bump peaks at 102 h−1 Mpc102\,h^{-1}\,\mathrm{Mpc}. The model is

ξmodel(s)=B ξt(αs)+a0+a1100s+a2(100s)2,\xi_{\mathrm{model}}(s) = B\,\xi_{\mathrm{t}}(\alpha s) + a_0 + a_1\frac{100}{s} + a_2\left(\frac{100}{s}\right)^2 ,

fitted over 5050 to 150 h−1 Mpc150\,h^{-1}\,\mathrm{Mpc}, ten bins, with the jackknife covariance and the Hartlap factor 0.6210.621. For each α\alpha from 0.800.80 to 1.201.20 the script solves for B≥0B \ge 0 and the broadband terms.

The block fit in the output gives the answer. The best α\alpha is 0.800.80, at the edge of the grid. The Δχ2=1\Delta\chi^2 = 1 interval covers the whole grid, so intervalTouchesGridEdge is true. The profile in Δχ2\Delta\chi^2 never rises above 0.150.15. Adding the template improves χ2\chi^2 from 1.081.08 to 0.890.89, a naive significance of 0.440.44. This patch leaves α\alpha free across the whole grid.

Compare with DESI

DESI 2024 III reports DV/rd=7.93±0.15D_V/r_d = 7.93 \pm 0.15 at zeff=0.30z_{\mathrm{eff}} = 0.30 for the full BGS sample. The script converts this with the fiducial cosmology, DVfid/rdfid=8.057D_V^{\mathrm{fid}}/r_d^{\mathrm{fid}} = 8.057, into αiso=0.984±0.019\alpha_{\mathrm{iso}} = 0.984 \pm 0.019, close to the 0.9829±0.01850.9829 \pm 0.0185 that the paper prints in its Table 15. The small difference comes from our use of z=0.295z = 0.295 where Table 18 prints the rounded 0.300.30, and from our CLASS value of rdfidr_d^{\mathrm{fid}}.

DESI's measurement used 300,017 galaxies over the whole DR1 footprint, an effective volume the paper gives as 1.7 Gpc31.7\,\mathrm{Gpc^3}, and reconstruction. The patch holds 15,699 galaxies in a geometric volume below 0.1 h−3 Gpc30.1\,h^{-3}\,\mathrm{Gpc^3}, which is 0.33 Gpc30.33\,\mathrm{Gpc^3}. For scale, the 2005 SDSS detection used 0.72 h−3 Gpc30.72\,h^{-3}\,\mathrm{Gpc^3} and reached 3.4σ3.4\sigma. The patch has too few pairs at 100 h−1 Mpc100\,h^{-1}\,\mathrm{Mpc}, and its own sample variance is large. That is the lesson of this guide: the BAO is a volume-limited measurement.

Four ways the measurement fails

Each failure case in the output changes one step and repeats the measurement.

Table 5.2 Change in s2ξs^2\xi relative to the correct measurement, from the expected output.
CaseAt s=85s = 85At s=105s = 105What went wrong
no-weights+32.9+32.9−8.3-8.3completeness and FKP weights dropped
one-third-of-the-randoms+50.6+50.6+66.3+66.3random catalogue cut to the galaxy count
unnormalised-pair-counts+3132+3132+4839+4839counts not divided by total pairs
distances-in-mpc-not-hinv-mpc+97.4+97.4+90.8+90.8distances in Mpc against an h−1 Mpch^{-1}\,\mathrm{Mpc} template

Dropping the weights shifts the curve by up to about 3333 inside the fit range, comparable to the errors. In a full survey, where errors are ten times smaller, that bias would dominate. Using too few randoms adds noise from the random catalogue itself. The changes of up to 6666 are within the errors here, and they grow as the random catalogue shrinks.

Forgetting the normalisation adds the constant offset (m−1)2/m2≈0.44(m-1)^2/m^2 \approx 0.44 derived in the pair-counting chapter. Multiplied by s2s^2 it grows into thousands.

Mixing units is the subtlest. With distances in Mpc, every separation is 1/h=1.481/h = 1.48 times larger, so the bin labelled 100100 holds pairs that are really at 67 h−1 Mpc67\,h^{-1}\,\mathrm{Mpc}, where ξ\xi is higher. Against a template in h−1 Mpch^{-1}\,\mathrm{Mpc}, a fit would read a dilation near 0.670.67 even with perfect data.

None of these fits bound α\alpha. In a larger sample, the last two would give wrong answers with small error bars.

What the numbers can support

The measured ξ(s)\xi(s) at separations below about 50 h−1 Mpc50\,h^{-1}\,\mathrm{Mpc} is a real, well-measured property of these galaxies. The covariance comes from 30 jackknife regions that are not identical in size, so it is itself uncertain at the tens-of-percent level. The fit result is only that the patch cannot bound α\alpha. The script has no reconstruction, no redshift-space modelling, no systematics weights beyond those in the catalogue, and a monopole-only fit. Those simplifications would matter for a real measurement. They do not change the conclusion here.

Sources: DESI 2024 III, sections 2 and 4 and tables 15 and 18, DESI DR1 data release paper, DESI DR1 large-scale structure catalogue documentation, DESI data licence and acknowledgment, Eisenstein et al. 2005, volume and significance of the first detection.

Export the Lab record and compare its galaxy count and error bars with the DESI patch output.

Sources

  1. DESI 2024 III, sections 2 and 4 and tables 15 and 18
  2. DESI DR1 data release paper
  3. DESI DR1 large-scale structure catalogue documentation
  4. DESI data licence and acknowledgment
  5. Eisenstein et al. 2005, volume and significance of the first detection

Keep reading

What this page links to directly

See this series on the content map →

Baryon acoustic oscillations · Reading path

From a sound wave to galaxy pair counts and DESI distances

  1. ArticleHow can pairs of galaxies measure the universe?
  2. ArticleHow far sound travelled before the drag epoch
  3. ArticleCounting galaxy pairs against a random catalogue
  4. ArticleReading the ruler along and across the line of sight
  5. You are hereMeasure a correlation function from a DESI data patch
  6. ArticleHow to read the DESI BAO results
Something broken, confusing, or missing on this page?Tell us
What kind of feedback?

Sent with this page’s address, its Lab ID, the site language and your device type. Please leave out personal details. Feedback is kept for up to 12 months.Privacy policy