Optics 040 · Imaging, instruments, and visual systems

Light-Field Camera & Display

Can one light-field photo be focused on either near or far objects, and what does that cost? Yes, after the shot. Shift the recorded views by different amounts and average them, and each shift brings a different depth into focus, though only one depth is sharp at a time. The cost is resolution: the sensor's pixels are shared among the angular views, so more views leave fewer pixels for each image.

A cached 4D ray dataset drives shift-and-average refocusing, replay of recorded views and a sub-aperture contact sheet. Opaque targets, a hole you can see through and a fixed total number of samples make occlusion, parallax that scales with inverse depth and the trade between spatial and angular samples visible.

Focus / near depth / replayed pupil

D,Z_n,u_{\mathrm{view}}

$0.50$

Angular views / spatial samples / unused samples

N_{uv},N_{xy},N_{\mathrm{unused}}

$50\%$

Geometric point-support spans and pupil span

b_n,b_f,B_{\mathrm{span}}

$0.00\pi$

Model regime $\text{valid model regime}$

Model checked against analytic results

Light-Field Camera & Display

Recorded views per pupil axis

N

5

Square capture aperture side (recapture)

B

140\,\mathrm{mm}

Post-capture focus depth (reuses the data)

D

550\,\mathrm{mm}

Near-target depth (moves the target and recaptures)

Z_n

550\,\mathrm{mm}

Horizontal replay (select a recorded view)

\xi_u

0

Focus on the cyan near target, then choose the far-focus preset: the same recorded rays now bring the gold rings into focus. Replay the horizontal views to see parallax and look through the hole. More angles leave fewer spatial samples. The dashed purple outline marks the chosen reconstruction plane, not a piece of glass.

Capture uses ideal point pupils and geometric resampling; this is not a commercial plenoptic camera or a lenticular display. Refocusing uses only the captured rays: no rays are generated after capture and no blur filter is added. Optical diffraction, finite-pupil integration and photon noise are not modeled. The contact sheet is rearranged and is not a raw microlens photograph.

Physics tutorial

Audit the physics of Light-Field Camera & Display

BackgroundAn ordinary photograph adds up the rays that pass through every part of the pupil. A light field keeps track of where each ray crosses the pupil and which way it travels: $L=L(u,v,q_x,q_y)$ . The first two coordinates give the position of a small point pupil, and the last two give the ray's slope toward the scene. Each pupil position records a complete view. The 3D apparatus is an ideal grid of parallel cameras, not a real microlens camera.

Why it mattersCan one light-field photo be focused on either near or far objects, and what does that cost?

Start with the essentials

Focus question: Can one light-field photo be focused on either near or far objects, and what does that cost?
One-sentence intuition: To refocus, shift each recorded view so that rays from the chosen depth line up, then average their linear radiance: $I_D(s,t)=\frac{1}{N^2}\sum_{i,j}L\!\left(u_i,v_j,s-\frac{u_i}{D},t-\frac{v_j}{D}\right)$ . Changing the focus depth captures no new rays. One shift cannot make near and far objects sharp together unless the pupil span shrinks to zero. A single point-pupil view, like the one replayed on the right, has no optical defocus.

Core mathematical model

Ray paths, parallax and occlusion

x(Z)=u+q_x Z,\quad y(Z)=v+q_y Z,\quad q_x=\frac{X-u}{Z},\quad \Delta q_x=-\frac{\Delta u}{Z}

The pupils sit at zero depth, and depth increases toward the scene. Each ray stops at the first surface it hits, so it meets the cyan target before the gold one. The dark cyan marks block light, and rays pass through the circular hole to the far target. When the pupil moves sideways, a nearer point shifts more. Rotating the overview is not the same as choosing a recorded view.

Why one depth lines up and another spreads out

s=\frac{X}{Z}+u\!\left(\frac{1}{D}-\frac{1}{Z}\right),\quad \Delta s=B_{\mathrm{span}}\left|\frac{1}{Z}-\frac{1}{D}\right|,\quad b=\frac{W\Delta s}{0.64}

When an object sits at the focus depth, the term that depends on pupil position vanishes. At other depths the separate pupils land their contributions at different places in the output. The readout gives this spread in pixels for an unblocked point. It leaves out bilinear interpolation width, clipping and occlusion, so it is not a measured image PSF. With few pupils, an out-of-focus point appears as separate copies instead of a smooth blur. With one view or zero baseline, refocusing has no effect that depends on depth.

More angular views leave fewer spatial pixels

W=\left\lfloor\frac{360}{N}\right\rfloor,\quad N_{uv}=N^2,\quad N_{xy}=W^2,\quad N_{\mathrm{used}}=(NW)^2\le360^2

More views along each pupil axis leave fewer spatial pixels in each view. When the view count does not divide the grid evenly, a few edge pixels go unused, and the readout lists them. The output images use the same spatial grid with a narrower, fixed angular crop, so displaying them larger adds no detail. The contact sheet lays out whole sub-aperture views. A raw plenoptic image is organized differently: it groups all the angles under each microlens, so the two kinds of picture cannot be swapped.

How the views are sampled and averaged

u_i=B\!\left(\frac{i+1/2}{N}-\frac12\right),\quad B_{\mathrm{span}}=B\frac{N-1}{N},\quad |q_x|,|q_y|\le0.65,\quad |s|,|t|\le0.32

The pupils sit at the cell centers of a square aperture. Choosing a view jumps to the nearest recorded column; with an even grid, the replay uses the row just below center. Refocusing resamples each view bilinearly in linear RGB, gives every pupil equal weight in the average and then applies a fixed sRGB encoding. No image gets its own contrast boost or an added blur kernel. Because the radiance is normalized, the model does not predict exposure, flux through the pupil area, noise or non-paraxial irradiance.

Common difficulties

Refocusing does not create new rays

Typical misconceptionRefocusing after the shot should bring back full-resolution detail and let me see through anything in the foreground.

Better mental modelRefocusing only rearranges rays that were already recorded. A partly hidden far point contributes only through the pupils that can actually see it, and a point hidden from every pupil cannot be recovered. Averaging can make a foreground blocker look fainter, but it does not make opaque material transparent. Each extra angular sample also takes pixels from the fixed spatial grid.

Run the experiment

01
Scene 1: Focus the near target
Switch between the near and far presets. The sharp cyan marks give way to sharp gold rings, while the recorded-view card and the captured data stay the same.
What to observe: Describing light by position and direction follows a 1996 paper; refocusing by shifting and adding sub-aperture views was derived in a 2005 paper. This Lab writes ray slopes in object space with its own stated coordinates instead of copying that paper's image-space depth parameter.
02
Scene 2: Refocus the far target
Replay the left and right views. Watch what shows through the real hole and how parallax changes with depth. Then set the aperture side to zero and both effects disappear.
What to observe: Only the aperture size, the number of angles and the near-object depth trigger a new capture. Focus and view replay work on the stored samples. The framing stays fixed so the images can be compared.
03
Scene 3: Dense angles, fewer spatial samples
Compare one, three and nine views per axis. Few views give separate out-of-focus copies; many views give coarse spatial pixels. These are two different artifacts. Then move the near object and refocus at its new depth.
What to observe: The display replays the angular data on an ordinary screen. It does not simulate a lenticular display or a microlens prescription, and it has not been validated against a real camera. Diffraction and integration over a finite pupil are left out.