What can twenty-six selected sources tell us about a population?

Specify the survey before counting its population

Consider a controlled survey covering five square arcminutes between redshifts 9.5 and 10.5. Its sources have physical Gaussian half-light radius 0.3 kiloparsecs, one prescribed screen and continuum slope, and fixed pixel noise. Measurement receives each true position and profile.

Changing the size of one source explores an individual measurement. A population calculation additionally needs a size distribution and a response appropriate to it. The response below keeps the original calibration fixed.

A selected source also enters a measured brightness bin. Different noise draws can put identical sources into neighbouring bins, or outside either end of the displayed range. Retaining those destinations preserves the probability accounting.

Reading: Sandbox Physics · Sandbox Physics

One response column describes all destinations of a source class

Divide intrinsic absolute magnitude into six bins. Density $\phi_j$ is constant within bin $j$ , in sources per magnitude per cubic megaparsec. Destination $i$ denotes eight selected categories or one nonselected category. Selected categories retain bright and faint overflow.

Integrate the probability of each destination over intrinsic magnitude and redshift. A response entry contains the available volume, intrinsic-bin width and destination probability. Its units already include magnitude and volume.

A_{ij}=\int_{M_j}^{M_{j+1}}\int_{z_1}^{z_2}P(i\mid M,z,\mathcal C)\,\frac{dV}{dz}\,dz\,dM

Condition

\mathcal C

includes the given spectrum, size, noise and selection. Survey area is included in the volume element. Response has units magnitude times cubic megaparsecs.

\sum_{i\,\mathrm{including\ nonselection}}A_{ij}=V_{\rm geom}\,\Delta M_j

Summing every destination recovers geometric volume times intrinsic-bin width. Summing just selected rows generally gives less.

Intrinsic magnitude interval	Width / mag
−22.5 to −21.5	1
−21.5 to −20.5	1
−20.5 to −19.5	1
−19.5 to −18.5	1
−18.5 to −17	1.5
−17 to −16	1

Reading: Sandbox Physics · Sandbox Physics

Send density through the response

Several intrinsic bins can feed one measured bin. Multiply each density by its corresponding response and add the contributions. Off-diagonal entries describe brightness migration. Dividing each observed bin by a single completeness number would omit that traffic.

The intrinsic widths are unequal and already integrated into the matrix. Do not multiply them again. Dividing a column by its intrinsic width gives an averaged effective volume for that source class; the matrix itself directly predicts measured counts.

The saved eight selected counts are zero, zero, zero, four, fourteen, eight, zero and zero: twenty-six in total. No observed nonselected count enters the fit. A nonselection destination in a calibration is different from knowing how many unseen objects actually occupied it.

\mu_i=\sum_{j=1}^{6}A_{ij}\phi_j

Use the eight selected rows. Density and response units cancel to an expected number.

Reading: Sandbox Physics · Sandbox Physics

Let empty bins contribute to the Poisson likelihood

Assume a Poisson source population whose objects independently enter categories with the calibrated probabilities. Independent thinning gives Poisson selected counts. A Poisson distribution describes nonnegative integer counts through one expected number.

The likelihood compares how probable the same observed counts are under different predicted means. Multiplying category probabilities and taking a logarithm gives a convenient sum. A larger value favours predictions that better explain the selected data.

\ell=\sum_i\left[n_i\ln\mu_i-\mu_i-\ln\Gamma(n_i+1)\right]

For zero observed count, the first term is taken as zero, but minus the expected count remains. A prediction of many sources in an empty bin is penalized. A positive observation with zero expected count has zero likelihood.

Reading: SciPy · Sandbox Physics · Sandbox Physics

Solve the population scale before its slope

Twenty-six selected sources do not constrain every arbitrary intrinsic-bin density equally well. Choose a restricted Schechter-shaped family with characteristic magnitude fixed at minus 20.6 and vary slope and normalization. Evaluate density at each intrinsic-bin midpoint and hold it constant within that bin.

The midpoint convention defines the fitted family. It differs from integrating a continuous Schechter function exactly within each bin. At a chosen slope, the response predicts counts for unit normalization. One common scale multiplies all those means, so its best value can be derived analytically.

q_j(\alpha)=0.4\ln10\,x_j^{\alpha+1}e^{-x_j},\qquad x_j=10^{0.4(M_\star-M_{j,\rm mid})},\quad\phi_j=\phi_\star q_j

Characteristic magnitude is fixed; slope lies between minus 3.5 and minus one. Normalization has units per cubic megaparsec.

b_i(\alpha)=\sum_jA_{ij}q_j(\alpha),\qquad \widehat\phi_\star(\alpha)=\frac{\sum_i n_i}{\sum_i b_i(\alpha)}

Differentiate with respect to normalization and set the derivative to zero. The selected total divided by the unit-normalization total gives its optimum. Total predicted and observed selected counts then agree; their distribution across bins still constrains slope.

Reading: Sandbox Physics · Sandbox Physics

Read the whole profile around the optimum

Substitute each slope’s best normalization into the likelihood to obtain the profile likelihood. The saved 401 points favour a slope about minus 2.004 and normalization about 0.0004435 per cubic megaparsec. Those values belong to the declared family and calibration.

A drop of 0.5 in log likelihood gives slope endpoints about minus 2.304 and minus 1.725. This describes the profile’s width and has an asymptotic interpretation under suitable large-sample conditions. It supplies no automatic exact coverage guarantee for these small counts. An endpoint reaching the predeclared family boundary must be marked truncated.

The faint intrinsic columns in the response largely flow into nonselection. These data have little direct leverage on them. A restricted family transfers information from brighter sources through its shape assumption. Freer faint densities need deeper measurements, more sources or independent information.

Each response column is divided by geometric volume and intrinsic width to show mean destination probability. The lower panel reads the saved profile; the dashed line marks a half-unit log-likelihood drop. No unseen source count is fitted. Columns denote discrete bins; a colour floor of 0.00001 affects display only.

Reading: Sandbox Physics · Sandbox Physics

Recompute the inference from a small download

The package needs selected counts, the complete response, bin edges and the original numerical functions. Extract it and run replay.py in the stated Python environment. It writes the full profile and checks, and refuses to overwrite an existing result so an earlier failure can be kept.

An actual cold run passed thirty-nine checks, including independent Poisson evaluation, a two-parameter optimum, scores, zero counts and invalid input. This replays inference through a fixed matrix. The original response has separate pixel/noise experiments; a real survey needs a calibration of its own observing and selection procedure.

Reading: Sandbox Physics · Sandbox Physics · Sandbox Physics

Try the observation

Read the individual source and fixed survey separately

Change a source’s size and follow its pixels and uncertainty.
Read the redshift, size, screen and area specified in the fixed survey panel.
Compare the complete response with the slope profile and explain the weak leverage on the faint intrinsic bins.

The population panel replays the fixed synthetic response and twenty-six selected counts. Individual controls calculate another specified source’s finite-pixel measurement.

Exercises and explained answers

What happens if the widest intrinsic bin is multiplied by another factor of 1.5?

Its contribution is counted 1.5 times too strongly, because its width is already integrated into the response. Tracking matrix units catches the duplication.

May an empty measured bin be removed from the likelihood?

No. If a model predicts many sources there, an empty observation is evidence against it. The Poisson log likelihood retains minus the expected count.

Double all selected counts without changing their relative distribution. How do the best slope and normalization change at fixed calibration?

The best slope stays the same and normalization doubles. Relative Poisson fluctuations decrease, generally narrowing the profile; evaluate the curve to check its width.

After changing one source’s screen, can the old matrix be called the new population response?

The matrix still describes the original population conditions. Specify the new screen distribution and recompute its response, or explicitly compare one changed source with the unchanged calibration.

Primary sources and calculation notes

Sandbox Physics, frozen synthetic response checks
Original 25-gate response record, including response refinement, finite Gaussian recovery and separate public-catalog count-identity checks. Its fixed source conditions do not calibrate a real survey.
Sandbox Physics, selected-only inference checks
Accepted 18-gate inference result: independent likelihood, optimum and scores. The first failed original attempt remains historical evidence; the public package uses only the accepted source and selected observations.
Sandbox Physics, selected-count cold replay
Complete nine-row/six-column calibration at two orders, eight selected counts, six unequal intrinsic bins, original numerical functions and all 401 profile points. Actual cold replay: 39 checks. No hidden parent truth or new draws.
SciPy, Poisson probability and logpmf
Probability mass function, nonnegative integer support and zero-mean case. The actual independent cold runs fix SciPy 1.17.1; documentation access does not update the execution environment.