The BAO bump lives in the two-point correlation function ξ(r), the fractional excess of galaxy pairs at separation r over a random distribution. A survey cannot count random pairs on paper, because its sky coverage and depth are patchy. So it builds a random catalogue with the same footprint and selection, counts data-data, data-random and random-random pairs, and combines them in the Landy-Szalay estimator (DD−2DR+RR)/RR. Errors come from repeating the measurement with one piece of the survey left out at a time. This chapter builds each step and shows what goes wrong when one is skipped.
Suppose galaxies have mean number density nˉ. The probability of finding one galaxy in a small volume dV1 and another in dV2, a distance r apart, is
dP=nˉ2[1+ξ(r)]dV1dV2.
If galaxies were scattered at random, ξ=0. Positive ξ means more pairs than random. In a homogeneous and isotropic universe ξ depends only on the distance r, not on position or direction. Redshift-space effects break the direction part, and the next chapter uses that.
Integrating over a thin shell of radius r and thickness Δr around every galaxy, the expected number of distinct pairs in a volume V holding N galaxies is
Npairs(r)≈2N(N−1)V4πr2Δr[1+ξ(r)].
Worked example. The Lab's default catalogue has N=6645 galaxies in a box 900Mpc on a side. For the shell from 145 to 150Mpc, the shell volume is 1.37×106Mpc3, a fraction 1.88×10−3 of the box. With 2.21×107 distinct pairs, about 41,400 fall in the shell if ξ=0, ignoring the box edges. The Poisson scatter is 41,400≈203, or 0.5 percent. A bump of a few tenths of a percent in one bin would be lost in that noise, which is why surveys need so many galaxies and combine many bins in a fit.
Why a random catalogue is needed
The formula above assumes a box with uniform coverage. A real survey has edges, holes around bright stars, regions observed more or less completely, and a number density that changes with redshift. All of these change the number of pairs at a given separation without any clustering at all.
The fix is to throw down random points that follow exactly the same footprint, mask and redshift distribution as the galaxies but have no clustering. Their pair counts measure the geometry alone. DESI ships random catalogues with each data release, with a weight column that encodes the same selection effects as the data. In the patch used in the method guide, one DESI random file held 1,043,189 points inside the patch, about 66 times the galaxy count.
Building the estimator
Let DD(r), DR(r) and RR(r) be the pair counts in a bin, each divided by its total number of pairs: ND(ND−1)/2, NDNR and NR(NR−1)/2 for distinct pairs. The simplest estimate compares data pairs with random pairs:
ξ^natural=RRDD−1.
Landy and Szalay proposed a combination that is less sensitive to the edges of the survey and has a smaller variance:
ξ^LS=RRDD−2DR+RR.
Eisenstein and colleagues used it for the first BAO detection in 2005, and DESI uses it for its correlation functions.
To see why it works, write the data density near the edge of a survey as nˉ(1+δ), where δ includes both real clustering and any mismatch between the true mean density and the one implied by the randoms. To first order DD/RR≈1+ξ+2δˉ and DR/RR≈1+δˉ, where δˉ is the average of δ around the bin. The natural estimator keeps the 2δˉ term. In the Landy-Szalay combination it cancels: 1+ξ+2δˉ−2(1+δˉ)+1=ξ.
Each count must be normalised. If they are not, the estimator picks up a constant offset. With raw counts and NR=mND randoms, DD:DR:RR≈1:m:m2 for ordered pairs, and
ξ^raw≈m21−2m+m2+m2ξ=m2(m−1)2+m2ξ.
Worked example. For m=3 the offset is 4/9=0.444. At s=95h−1Mpc this adds about 0.444×952≈4000 to s2ξ. The method guide's unnormalised failure case shows an offset of 3945 there, slightly less because the weighted sums do not stand in an exact ratio of 3.
Pairs can carry weights. DESI multiplies a completeness and systematics weight by an FKP weight, named after Feldman, Kaiser and Peacock, that gives each galaxy the importance that maximises the signal-to-noise at BAO scales. Then each pair counts with wiwj, and the normalisations become (∑w)2−∑w2 for the auto-counts and ∑wD∑wR for the cross-count.
Why plots show s squared times xi
The correlation function falls steeply. In the DESI patch it drops from 1.87 at 5h−1Mpc to about 0.005 near 100h−1Mpc. On a linear plot the bump would be invisible. Multiplying by s2 roughly flattens the falling part, so a feature near 100h−1Mpc stands out. Here s is the separation in redshift-space coordinates, computed with an assumed distance-redshift relation. It equals r only if that assumption is right.
Errors from the survey itself
Neighbouring bins share galaxies, so their errors are correlated. A standard way to estimate the full covariance from the data alone is the jackknife. Split the survey into Njk regions of similar size. For each k, remove region k and recompute ξ^k. Then
The factor Njk−1 makes up for the fact that the leave-one-out samples overlap almost completely.
A fit needs the inverse covariance. An inverse built from a noisy estimate is biased high. For Gaussian errors, Hartlap and colleagues showed that multiplying by (Njk−p−2)/(Njk−1), with p bins, removes the bias. With 30 regions and 10 bins this factor is 18/29=0.621. When p approaches Njk, the estimate becomes useless. The Lab avoids that problem by using only the diagonal of its 27-region jackknife, which ignores the correlations between bins. DESI uses analytic and mock-based covariances instead.
In the DESI patch, neighbouring bins inside the fit range have correlation coefficients up to 0.92, and some distant pairs of bins are anticorrelated, down to −0.58. With only 30 regions these coefficients are noisy themselves. A fit that treated the bins as independent would misjudge its precision.
Lab tasks
In the default catalogue, find the bin where the transverse wedge peaks. Compare its height with the error bar on a neighbouring bin.
Open the Too faint to see by eye preset. Read the naive detection and compare its error bars with the default.
Open the No sound wave at all preset. Does any bump survive? What does the fitted stretch do?
Exercises
Repeat the worked example for the shell from 95 to 100Mpc. How many random pairs are expected, and what is the fractional Poisson scatter?
In one bin the normalised counts are DD=1.012, DR=1.004 and RR=1.000. Compute both estimators and say which one is more trustworthy near an edge.
A student uses raw counts with ten times more randoms than galaxies. What constant offset appears in ξ^?
Compute the Hartlap factor for 27 regions and 10 bins, and for 27 regions and 25 bins. What do you conclude?
AnswersShow the answer
The shell volume is 4π×97.52×5=5.97×105Mpc3, a fraction 8.19×10−4 of the box. Then 2.21×107×8.19×10−4≈18,100 pairs, with scatter 18,100≈134, or 0.74 percent. Smaller shells hold fewer pairs and are noisier.
Natural: 1.012−1=0.012. Landy-Szalay: (1.012−2.008+1)/1=0.004. The data-random count is 0.4 percent high, a sign that the mean density near this bin differs from the random catalogue. Landy-Szalay removes that first-order term, so 0.004 is the better estimate.
With m=10, the offset is (10−1)2/102=0.81.
(27−10−2)/26=0.577 and (27−25−2)/26=0. With 25 bins and 27 regions the inverse covariance carries no usable information, so either use fewer bins, more regions, or a different covariance.