Optics 040 · Imaging, instruments, and visual systems

Light-Field Camera & Display

A cached 4D ray dataset drives real shift-and-average refocusing, recorded-view replay and a sub-aperture contact sheet. Opaque targets, a genuine hole and a fixed sample budget make occlusion, inverse-depth parallax and spatial-angular exchange visible.

Focus / near depth / replayed pupil

D,Z_n,u_{\mathrm{view}}

$0.50$

Angular views / spatial samples / unused budget

N_{uv},N_{xy},N_{\mathrm{unused}}

$50\%$

Geometric point-support spans and pupil span

b_n,b_f,B_{\mathrm{span}}

$0.00\pi$

Model regime $\text{valid model regime}$

Ideal point-pupil capture and geometric resampling, not a commercial plenoptic camera or lenticular display. No post-capture ray regeneration, hidden blur filter, optical diffraction, finite-pupil integration or photon-noise model. A rearranged contact sheet is not a raw microlens photograph.

Physics tutorial

Audit the physics of Light-Field Camera & Display

BackgroundA photograph merges rays arriving from different pupil positions. A light field keeps both position and direction: $L=L(u,v,q_x,q_y)$ . Here the first two coordinates locate a point pupil; the last two are ray slopes toward the scene. Each pupil records a complete view. The 3D rig is an ideal parallel-camera sampling construction, not a literal microlens camera.

Why it mattersCan the same recorded rays bring near and far objects into focus—and what detail do angular views cost?

Start with the essentials

Focus question: Can the same recorded rays bring near and far objects into focus—and what detail do angular views cost?
One-sentence intuition: Refocusing shifts the recorded views so rays meeting at the chosen depth agree, then averages their linear radiance: $I_D(s,t)=\frac{1}{N^2}\sum_{i,j}L\!\left(u_i,v_j,s-\frac{u_i}{D},t-\frac{v_j}{D}\right)$ . No new scene rays are captured when this control moves. Near and far cannot both become sharp through one shift unless the pupil span vanishes. The right-hand replay shows one recorded point-pupil view, which has no optical defocus.

Core mathematical model

Ray geometry, parallax and actual occlusion

x(Z)=u+q_x Z,\quad y(Z)=v+q_y Z,\quad q_x=\frac{X-u}{Z},\quad \Delta q_x=-\frac{\Delta u}{Z}

Pupils lie at zero axial depth; object depth is positive toward the scene. First-hit testing meets the cyan target before the gold target. Cyan dark marks are opaque, while the circular hole passes rays to the far target. A nearer point moves more with pupil translation. Orbiting the overview is not selecting a recorded view.

Why one depth aligns while another spreads

s=\frac{X}{Z}+u\!\left(\frac{1}{D}-\frac{1}{Z}\right),\quad \Delta s=B_{\mathrm{span}}\left|\frac{1}{Z}-\frac{1}{D}\right|,\quad b=\frac{W\Delta s}{0.64}

At matching depths the pupil-dependent term is zero. Elsewhere the discrete contributions spread across the output. The readout reports this unoccluded point-support span in pixels; it excludes bilinear interpolation width, clipping and occlusion, so it is not a measured image PSF. Sparse pupils can make separate copies rather than smooth blur. One view or zero baseline has no depth-dependent refocus effect.

A finite data budget, not free super-resolution

W=\left\lfloor\frac{360}{N}\right\rfloor,\quad N_{uv}=N^2,\quad N_{xy}=W^2,\quad N_{\mathrm{used}}=(NW)^2\le360^2

Increasing views per pupil axis decreases captured spatial pixels per view. Non-divisors leave a small unused edge budget, reported explicitly. Output images reuse the same spatial grid with a narrower fixed angular crop; enlarging their display does not invent detail. The contact sheet rearranges entire sub-aperture views. Raw plenoptic microimages instead group angles under each spatial microlens; they are not interchangeable pictures.

Fixed sampling, fixed exposure, honest replay

u_i=B\!\left(\frac{i+1/2}{N}-\frac12\right),\quad B_{\mathrm{span}}=B\frac{N-1}{N},\quad |q_x|,|q_y|\le0.65,\quad |s|,|t|\le0.32

Sampling uses cell centers on a square aperture. View selection snaps to a recorded column; even grids use the lower central row. Focus averages all pupils equally after bilinear spatial resampling in linear RGB, then applies fixed sRGB encoding. There is no per-image contrast boost or arbitrary blur kernel. This normalized radiance construction does not predict exposure, pupil-area flux, noise or non-paraxial irradiance.

Common difficulties

A depth slider can recover rays never recorded

Typical misconceptionPost-capture focus should recover full-resolution detail and see through any foreground object.

Better mental modelRefocusing only reorganizes recorded rays. Occluded far points contribute only through pupils that actually see them; no pupil can recover a completely hidden point. Averaging may soften foreground occlusion but does not turn opaque material transparent. More angular samples consume the fixed spatial budget.

Run the experiment

01
Scene 1: Focus the near target
Compare near and far presets. Watch sharp cyan marks give way to sharp gold rings while the recorded-view card and capture remain unchanged.
What to observe: The 1996 paper motivates position-direction light-field representation; the 2005 paper derives sub-aperture shift-and-add refocusing. Our object-space slopes use a stated coordinate convention rather than copying its image-space depth parameter.
02
Scene 2: Refocus the far target
Replay left and right views. Track the real opening and depth-dependent parallax; then set the aperture side to zero and observe both effects vanish.
What to observe: Only changing aperture size, angular count or near-object depth recaptures. Focus and view replay transform stored samples; fixed framing makes changes comparable.
03
Scene 3: Dense angles, fewer spatial samples
Compare one, three and nine views per axis. Sparse defocused copies and dense coarse spatial pixels are different artifacts. Move the near object and refocus to its new depth.
What to observe: The display replays angular data on an ordinary screen. It is not a lenticular optical display, a microlens-prescription simulation or experimental camera validation. Diffraction and finite-pupil integration remain outside scope.