Recover a spectrum after detector smearing

A detector can miss an event or place it in the wrong momentum bin. A response matrix describes both effects. Unfolding uses that matrix and the observed counts to infer a particle-level distribution. The difficulty is that smearing can erase the very differences we want to recover: small fluctuations in the measured counts may become large oscillations in the inferred spectrum.

Generate the record before reconstructing the distribution

Label particle-level bins by $j$ and reconstructed bins by $i$. Let $x_j$ be the expected number of selected-process events before detector losses. The response element $R_{ij}$ is the probability that a true event in bin $j$ is recorded in reconstructed bin $i$. A column sum gives that true bin’s efficiency, which can be smaller than one.

Matrix multiplication is a collection of weighted sums: each reconstructed bin receives contributions from every true bin. The background expectation $b_i$ is added after the signal has passed through the response. For an unweighted counting experiment, the recorded count in each bin can then be drawn from a Poisson distribution with mean $\nu_i$.

A reproducible simulation must save the event-level truth and its reconstructed outcome, including missed events, or an equivalent sufficient count representation. The histogram used in reconstruction comes from those saved outcomes. An ideal curve can be displayed alongside it as a separate reference.

Solve a two-bin example exactly

Consider two true bins, each recorded with 80 percent efficiency. Of all true events, 60 percent stay in the corresponding reconstructed bin and 20 percent migrate to the other bin. The remaining 20 percent are missed. Take zero background and a true mean vector containing 100 and 50 events.

The total reconstructed expectation is 120, equal to 80 percent of the true total of 150. Direct inversion returns the input exactly for these noiseless means. Now imagine a fluctuation that adds one event to the first reconstructed bin and removes one from the second. The inverse maps that change into an increase of 2.5 and a decrease of 2.5 in the true bins.

Identify the information that smearing erases

The response transmits the sum mode by a factor 0.8 and the difference mode by 0.4. Inversion therefore amplifies them by 1.25 and 2.5. If the two response columns become nearly identical, the difference mode is almost erased and its inverse becomes enormous. This independent analytic limit explains the instability without relying on a particular unfolding program.

Stabilize the answer, then measure the cost

Regularization discourages features that the detector cannot resolve reliably. A smoothness penalty, a truncated inverse mode or a limited number of iterations can all suppress unstable fluctuations. They also bias some possible truths. A narrow feature may become smoother than it really is.

Reassign the observed counts iteratively

One iterative construction starts with positive true-bin weights $x_j^{(0)}$. For a background-free example, it uses Bayes’ rule to assign each observed bin probabilistically to possible true bins, then corrects for efficiency:

Here $k$ is the iteration number and $n_i$ the saved reconstructed count. Zero-efficiency bins cannot be recovered from these data. Terms with zero predicted reconstructed expectation require an explicit treatment; they cannot be fixed by dividing by an arbitrary tiny number. More iterations reduce some dependence on the starting shape but can expose more statistical noise. The number of iterations is part of the estimator definition.

A closure test and a held-out test answer different questions

Generate and unfold with the same response and truth family, and the mean result should recover that family within the estimator’s expected bias. This closure test catches mistakes. It is easier than the real problem because the generating assumptions match the reconstruction assumptions.

For a stronger check, hold out a shifted peak, a different slope or a changed response width. Reconstruct those samples using the frozen nominal response and stopping rule. Compare bias with the reported uncertainty, and repeat pseudoexperiments. Keep the training sample, test sample and random seeds separate. A result can refold well and still be biased in a poorly measured true-space direction.

Work with the detector Lab

Start with the detector-unfolding Lab’s saved default synthetic sample. Identify the truth reference, reconstructed counts and inferred distribution from their labels. Increase the resolution width alone and predict which response columns become harder to distinguish. Then vary the iteration setting and compare the noise with the recovered feature width. The saved default has two truth peaks centred at 82 and 110 GeV, intrinsic width 4.5 GeV, detector resolution 6 GeV and eight iterations. At seed 330031, 1234 of 1500 generated events enter the reconstructed histogram; efficiencies and mass-range losses act before that histogram is made.

For the second comparison choose “Calibration misses a shift”, which adds a 6 GeV shift only to the acquisition response. Record what changed in the generator and what stayed fixed in the reconstruction. Changing only the iteration count reuses the saved reconstructed records. Changing the detector resolution or acquisition preset generates a new deterministic sample with the displayed seed; the inference then uses those newly saved outcomes.

Exercises and explanations

Exercise 1. Divide the reconstructed means 70 and 50 by efficiency 0.8. Does this recover the true bins?

Answer. It gives 87.5 and 62.5. Their sum is correct, but their shape is wrong because events migrated between bins. An efficiency correction alone corrects losses, not migration.

Exercise 2. What can be learned about a true bin whose response column is entirely zero?

Answer. The selected data supply no information about its rate. An inferred value must come from an external constraint or model, and should be labelled accordingly. Adding more selected events elsewhere does not resolve this missing direction.

Exercise 3. In the two-bin example, what happens as the same-bin and migration probabilities both approach 0.4?

Answer. The columns become identical. Both reconstructed bins then depend only on the true total, so their difference is unidentifiable. The inverse singularity expresses a physical loss of information.