Skip to main content
Sandbox Physics
Direct imaging, from glare to a spectrum

L3 · Direct imaging

Recover a spectrum without losing its uncertainty

Sandbox Physics ·

A recovered spectrum is a set of source contrasts that explains the processed image measurements through their response matrix. Its uncertainty depends on which spectral combinations survive, how well the source position is known and how the noise was estimated.

Fit the measured combinations of colors

Collect the five processed aperture sums into a vector. Injecting one entrance band at a time gives five response columns. Multiplying the response matrix by a proposed contrast vector predicts those measured sums. Add a common local background vector because a subtraction residual can have a nonzero mean around the star.

The comparison apertures estimate that background. For a known common covariance and independent comparison patches, subtracting their mean removes the unknown background while adding the mean-estimation variance. This is also the result of profiling a shared background in a joint Gaussian model. The actual ring is only an approximation to those independent patches, which we will test.

A two-band example shows why the complete matrix matters. Suppose each band retains its own source but subtracts half the other band. Equal true contrasts of one then produce outputs of one half in both bands. Dividing by the diagonal would report one half; solving the coupled equations recovers one in both. This example is dimensionless teaching arithmetic, separate from the sampled five-band instrument.

y=Ac+b+ϵ,Cdiff=(1+1/nref)Cring+Csource\boldsymbol y=A\boldsymbol c+\boldsymbol b+\boldsymbol\epsilon,\qquad C_{\rm diff}=(1+1/n_{\rm ref})C_{\rm ring}+C_{\rm source}
The response converts dimensionless contrast to measured counts. The background comparison count is the number of resolution patches, not the number of pixels. The added source covariance is absent from source-free patches.
(1−1/2−1/21)(11)=(1/21/2)\begin{pmatrix}1&-1/2\\-1/2&1\end{pmatrix}\begin{pmatrix}1\\1\end{pmatrix}=\begin{pmatrix}1/2\\1/2\end{pmatrix}
Off-diagonal mixing can alter a broad common spectral level even when the diagonal response appears simple.

Reading: Pueyo · Greco & Brandt

Some spectral combinations are easier to measure

Covariance records which fluctuations move together. If two channels have unit variance and correlation 0.8, their shared-brightness combination has covariance eigenvalue 1.8, while their difference has eigenvalue 0.2. A broad rise and a color difference therefore carry different noise weights.

The response has its own preferred combinations. Singular values measure how strongly it maps independent input combinations into measured output. The saved ADI-plus-SDI response has singular values from about 0.706 million to 4.63 million electrons per unit contrast. Its weakest combination is still measurable in this example, but it is much less strongly transmitted.

With zero field rotation, the mean-ADI coadd has zero source response. No inverse can recover the removed information. A rank check should report that failure. Adding a numerical ridge can produce a smooth spectrum, but the ridge supplies extra assumptions and changes the inference.

∂χ2∂c=−2ATC−1(y−bˉ−Ac)=0\frac{\partial\chi^2}{\partial\boldsymbol c}=-2A^{\mathsf T}C^{-1}(\boldsymbol y-\bar{\boldsymbol b}-A\boldsymbol c)=0
For a fixed covariance, expand the quadratic residual and set its derivative with respect to each contrast to zero. Moving the model term to the other side gives the normal equations used below.
c^=(ATC−1A)−1ATC−1(y−bˉ)\widehat{\boldsymbol c}=(A^{\mathsf T}C^{-1}A)^{-1}A^{\mathsf T}C^{-1}(\boldsymbol y-\bar{\boldsymbol b})
This unrestricted generalized least-squares estimate assumes a fixed response and positive-definite covariance. Negative channel estimates remain visible. Singular normal matrices are rejected.
Five by five measured spectral response with positive diagonal and negative off-diagonal entries.
Measured five-band response of the same synthetic ADI-plus-SDI acquisition, normalized by the stellar entrance count. Rows are measured output, columns are separately injected input. Negative off-diagonal terms encode spectral subtraction.

Reading: Pueyo · Greco & Brandt

Let position and brightness compete

A slightly misplaced PSF can mimic a change in brightness. One aperture sum per band hides much of that ambiguity: five measurements and five freely fitted contrasts have no residual degrees of freedom. To ask whether the image shape agrees, add spatial measurements.

The saved position calculation uses three measurements per band: a central aperture, a radial aperture difference and a tangential aperture difference. There are fifteen measurements in total. For every trial position, the point-source field is propagated again, injected channel by channel and processed again. Five contrasts are fitted at each position. The remaining residuals test more than total flux.

Comparison patches supply the full fifteen-by-fifteen spatial and spectral covariance. The default geometry leaves seventeen patches after excluding the companion and its negative ADI arc. Their sample covariance has rank at most sixteen, so rank fifteen is possible, but its entries can still be poorly determined. In the sixty-four repeated angular-subtraction acquisitions, the median comparison-covariance condition number was about seventy thousand; the largest exceeded nine million.

The condition number is the largest covariance eigenvalue divided by the smallest; a large ratio makes inversion sensitive to errors in weak directions. A profile retains the best-fitting contrasts at each position. It is a way to see position–spectrum trade-offs. The coarse saved nine-position map is useful for that diagnostic, but its grid is too sparse to quote a precise position interval.

Reading: Pueyo · Greco & Brandt · Mawet et al.

Count the source photon noise too

A source-free ring can estimate residual halo fluctuations, but it has none of the companion photons. A bright recovered source therefore needs an additional photon-noise term. Omitting that term in the saved default spatial fit gives a residual quadratic form about 325 for ten residual dimensions. The independent reproduction saves that failed calculation too.

For mean ADI and mean SDI, work backwards through the actual interpolation and subtraction to obtain the weight of each input pixel in each measurement. Multiplying pairs of weights by the source photon variance and summing gives its full covariance. Different output channels can share input counts after SDI. The fitted nonnegative source intensity supplies this variance model; the reported contrast estimates themselves remain unrestricted.

The corrected default spatial fit has a quadratic residual about 5.42. This is one check of a conditional model. Data-dependent KLIP needs more: source-photon resamplings rebuild the reference basis, and complete independent acquisitions re-estimate the comparison covariance. Fixed-basis propagation alone conditions away those changes.

(Csource)ij=∑tℓpwi,tℓpwj,tℓp μsource,tℓp(C_{\rm source})_{ij}=\sum_{t\ell p}w_{i,t\ell p}w_{j,t\ell p}\,\mu_{{\rm source},t\ell p}
For a fixed linear numerical operator, the weights are its transpose acting on the output measurement. This is exact source-photon variance propagation for that operator, conditional on the adopted source intensity. Learning the KL basis and estimating the comparison covariance add uncertainty tested by complete new acquisitions.

Reading: Greco & Brandt · Pueyo

Write down the likelihood that is actually being profiled

A multivariate Gaussian density contains both a residual penalty and a normalization. The normalization depends on the determinant of the covariance. If the noise model changes with source flux or trial position, that determinant changes too. Minimizing only the weighted residual can then favor an inflated noise model.

The local calculation estimates source photon covariance from a preliminary flux fit, then holds the combined covariance fixed across the position grid. Its determinant is included in the saved Gaussian working likelihood and is constant within that grid. This is a conditional profile with an adopted covariance. Updating flux-dependent covariance during optimization would require updating the determinant at every step.

A posterior would additionally require a specified prior and integration over nuisance quantities. A profile keeps best-fitting values instead. The two objects answer different questions, particularly near weak-response directions or a nonnegative-flux boundary.

−2log⁡L=rTC−1r+log⁡det⁡C+mlog⁡(2π)-2\log L=\boldsymbol r^{\mathsf T}C^{-1}\boldsymbol r+\log\det C+m\log(2\pi)
The residual is measurement minus model, and m is the measurement dimension. Count units are fixed throughout, so the same density convention is used at every grid point.

Reading: Greco & Brandt · Pueyo

Ask repeated acquisitions how the errors behave

For each repetition, generate fresh Poisson counts and read noise from the same optical expectation. Rebuild the processed images, exclude the same geometric source arc, re-estimate the comparison covariance and run the spatial–spectral fit. The source input is known here, so a nominal one-standard-error interval can be checked against that truth.

The table reports coverage counts in wavelength order from sixty-four ADI or ASDI acquisitions and twenty-four KLIP acquisitions. A Gaussian one-standard-error interval would cover about sixty-eight percent in an ideal known-covariance problem. These tests retain the fluctuations, sparse-covariance errors and the deterministic adopted halo. They do not generate a new real atmosphere.

A finer position grid, spaced by 0.001 reference angular units, also reveals a limit hidden by the coarse map: minima reach the edge in 22 of 64 ADI repetitions and 31 of 64 ASDI repetitions. The 24 KLIP repetitions have fifteen boundary minima. These grids are local diagnostics. Boundary minima call for a wider search, followed by a separate position-interval coverage check.

The shortfall in some channels matters when interpreting a narrow spectral feature. The uncertainty model is useful, yet its nominal interval is not automatically a calibrated interval. A PSF mismatch test asks a further question by generating light with one PSF and fitting another. More noise repetitions cannot repair an incorrect optical template.

ProcessingWithin one standard errorWithin two standard errors
Mean ADI, 64 acquisitions48, 44, 41, 38, 3964, 62, 57, 61, 56
SDI plus ADI, 64 acquisitions44, 46, 35, 43, 3463, 60, 60, 61, 61
Data-dependent KLIP, 24 acquisitions11, 15, 8, 12, 1119, 22, 23, 19, 18

Reading: Greco & Brandt · Mawet et al.

Separate a local fluctuation from a blind search

A local Student statistic compares one aperture with a finite reference sample. With only five independent Gaussian reference values, a statistic of five has a one-sided tail probability about 0.00375, far above the familiar large-sample Gaussian value. Correlated speckles can change that distribution further.

Searching many positions and trying several subtraction ranks adds opportunities for a large fluctuation. A null experiment repeats the complete position and method selection and saves its overall maximum. Even zero exceedances in a finite null simulation imply an upper bound, never a measured probability of zero.

The practical endpoint is to report the recovered combinations, the response, the noise assumptions and the observed failures together. They tell the next observer whether more rotation, a better PSF or a longer sequence is likely to help.

t=x−xˉrefsref1+1/nref,ν=nref−1t=\frac{x-\bar x_{\rm ref}}{s_{\rm ref}\sqrt{1+1/n_{\rm ref}}},\qquad\nu=n_{\rm ref}-1
The nominal Student reference assumes independent, equal-variance Gaussian resolution elements. Degrees of freedom count independent reference elements minus one.

Reading: Mawet et al. · Greco & Brandt

Try the observation

Inspect a spectrum and its weak directions

  1. Compare the five-band spectrum with the prescribed dashed input and the full response matrix.
  2. Reduce total rotation towards zero. Explain any identifiability notice using the mean-ADI equation.
  3. Compare the deceptive-halo preset with the default. Decide whether the spectrum or its uncertainty is driving your interpretation.

Exercises and explained answers

You fit five freely varying channel contrasts to five aperture sums and obtain a residual quadratic form near zero. What did that test establish?

A nonsingular square response can fit those five numbers exactly, even with a poor PSF or noisy data. It checked an algebraic solve, not goodness of fit. Add independent spatial measurements or held-out data, then examine their residuals.

A fifteen-dimensional sample covariance from seventeen patches has full rank. Is that enough to trust a narrow interval?

Full rank permits inversion but says little about how accurately its small eigenvalues were estimated. The covariance has only sixteen centered sample directions and can be badly conditioned. Repeated-acquisition coverage, sensitivity to patch selection and PSF mismatch directly test the interval.

A fit changes its photon variance when source flux changes but drops the determinant term. How would you diagnose the consequence?

Compare the complete Gaussian likelihood with the residual-only objective over the same flux grid. An increased variance can lower the residual penalty without being penalized for spreading probability over more data values. The logarithmic determinant supplies that normalization.

No null trial out of sixty-four exceeds the selected peak. What one-sided 95-percent binomial upper bound can you report?

Solve (1−p)64=0.05(1-p)^{64}=0.05 to obtain p<1−0.051/64≃0.0457p<1-0.05^{1/64}\simeq0.0457. This bound applies to that simulated selection and population. At this sample size the experiment resolves percent-scale rates.

Sources and calculation scope

  1. Pueyo, Detection and Characterization of Exoplanets Using Projections on Karhunen–Loève Eigenimages (2016)

    Fixed v1, Sections 2.1–2.5, 3.1–3.3 and 4.1–4.2; Appendix E, Equations E18/E20, and Appendix F, Equations F8–F12. Source contamination changes the reference basis. The illustrative J-band spectra use synthetic injections.

  2. Mawet et al., Fundamental Limitations of High Contrast Imaging Set by Small Sample Statistics (2014)

    Fixed v1, Sections 3.1–3.3 and 4.1, Equations 7–10. Read the resolution-element count and assumptions before converting a statistic into a tail probability.

  3. Greco & Brandt, The Measurement, Treatment, and Impact of Spectral Covariance and Bayesian Priors in Integral-Field Spectroscopy of Exoplanets (2016)

    Fixed v1, Section 2, Equations 1–6; Sections 3.1–3.2, Equations 7–8. Spectral covariance changes the weighting of broad spectral structures.

FeedbackSomething broken, confusing, or missing on this page?Tell us
What kind of feedback?

Sent with this page’s address, its Lab ID, the site language and your device type. Please leave out personal details. Feedback is kept for up to 12 months.Privacy policy