How does a camera measure motion?

Pixels do not tell us velocity directly. A camera records only which pixel column the object falls on in each frame. To get a velocity, we first calibrate pixels into positions, then estimate a slope from how the position changes over a few neighbouring frames. The result carries the effects of the motion, the camera and the analysis method all at once, so each has to be checked separately.

In the last chapter the position curve was given to us. Now the question changes. A fixed camera films a straight track at 30 frames per second. A ball oscillates back and forth along the track while drifting slowly to the right. All we have is the pixel column in each frame. Can we read the ball's velocity at each moment from those numbers?

Every number in this chapter comes from the Motion Measurement and Numerical Differentiation Lab. There the motion is given by a known formula, so estimates can be compared with the truth. A real experiment has no such truth to compare against, only the checks this chapter introduces.

2.1 What the camera records

The camera takes frames at a fixed frame rate $f_s$, so neighbouring frames are $\Delta t=1/f_s$ apart. Counting from zero, frame $n$ is taken at $t_n=n\,\Delta t$. At $30\,\mathrm{Hz}$ the frame interval is about $0.033\,\mathrm s$.

In each frame, a tracking program finds the pixel column $u_n$ of the ball's centre. A record is a list of number pairs $(t_n,u_n)$. It contains no positions and certainly no velocities: positions come from a calibration, and velocities from a calculation.

This distinction runs through the whole chapter. What is recorded directly is an observation; what we compute from observations is an estimate. An estimate's error comes partly from the observations and partly from the algorithm we chose.

The Lab's default record lasts $4\,\mathrm s$ and has 121 frames. The sensor is 2048 columns wide, and column 1024 is lined up with the origin of the track. The ball's true motion is

The defaults are $x_b=0$, $b=0.15\,\mathrm{m\,s^{-1}}$, $c=0$, $A=1.5\,\mathrm m$, $f=0.5\,\mathrm{Hz}$ and $\phi=0$. The frequency $f$ counts oscillations per second; $\omega$ is the angular frequency, in radians per second, and the sine takes its argument in radians. The ball goes back and forth once every two seconds while drifting right at $0.15\,\mathrm{m\,s^{-1}}$.

2.2 The velocity of a sinusoidal motion

To compare estimates with the truth we need this motion's velocity. Chapter 1 found slopes of polynomials only, so here we use a symmetric average velocity to find the slope of a sine.

Take two times a step $h$ either side of $t$. The identity $\sin(\alpha+\beta)-\sin(\alpha-\beta)=2\cos\alpha\sin\beta$ gives

As $h$ goes to zero, $\sin(\omega h)/(\omega h)$ goes to 1. This is the small-angle fact that the sine of a small angle is almost equal to the angle in radians; for example, $\sin(0.01)=0.0099998$. So

The default motion's top speed is about $A\omega=1.5\pi\,\mathrm{m\,s^{-1}}\approx4.71\,\mathrm{m\,s^{-1}}$. The ratio $\sin(\omega h)/(\omega h)$ will return in Section 2.5, where it sets how much a difference underestimates the velocity.

2.3 Calibration: turning pixels into positions

If the camera scale is $s$ pixels per metre and the origin sits at column $u_0$, then

In the Lab, $s=120\,\mathrm{px\,m^{-1}}$ and $u_0=1024$, so one pixel corresponds to about $8.3\,\mathrm{mm}$. In a real experiment you usually photograph a ruler of known length, count how many pixels it spans, and get $s$ from that.

Calibration can go wrong in two ways, with very different effects.

A wrong scale stretches every position by the same factor. If the assumed scale is 10% larger than the true one, every position, displacement and velocity is divided by 1.1. The error is proportional to the quantity being measured: the further the ball goes, the larger the position error.

A wrong origin shifts every position by the same distance. An origin 12 pixels off moves every position by $12/120\,\mathrm m=0.1\,\mathrm m$. The shift cancels when two positions are subtracted, so velocities are unaffected.

The Lab's “Uniform motion” preset moves the ball at a steady $0.5\,\mathrm{m\,s^{-1}}$ with no noise. With a 10% scale error, the velocity error is exactly $0.5\times(1-1/1.1)\,\mathrm{m\,s^{-1}}=0.0455\,\mathrm{m\,s^{-1}}$. With a 12-pixel origin error, the position error is $0.1\,\mathrm m$ and the velocity error is zero. Both results follow from the formula above without the Lab.

2.4 Summarising a list of errors

A record has more than a hundred estimates, each with an error $e_k$. To compare two methods we need to sum the list up in a few numbers.

The mean says whether the errors lean to one side. The standard deviation says how widely they spread around the mean. The root mean square (RMS) includes both:

Write $e_k$ as $(e_k-\bar e)+\bar e$, square and sum. The cross term sums to zero, because the deviations $e_k-\bar e$ cancel in total. Hence

With the origin 12 pixels off, the position errors have a mean of $0.1\,\mathrm m$, a standard deviation of zero and an RMS of $0.1\,\mathrm m$. Every RMS error the Lab reports is the RMS of the difference between an estimate and the known truth, which only a simulation can compute directly.

The same argument shows something else. When two unrelated errors add, the cross term averages to zero, so their standard deviations do not add directly: their squares do. This chapter uses that rule several times.

In every frame the Lab adds a perturbation between $\pm0.5$ pixels and then rounds the result to a whole pixel. The perturbation comes from a fixed formula, so it is not truly random noise, but over many frames it spreads roughly evenly across that interval. A quantity spread evenly between $\pm a$ has a standard deviation of $a/\sqrt3$. We use that result here without proof; deriving it needs the integral of $x^2$.

The perturbation's standard deviation is $0.5/\sqrt3\approx0.29$ pixels. The rounding error lies between $\pm0.5$ pixels too, with a standard deviation of about 0.29 pixels. Adding the squares and taking the square root gives about 0.41 pixels, or $0.0034\,\mathrm m$. The Lab's raw position RMS error for the default record is $0.00336\,\mathrm m$, in line with the estimate.

2.5 Differences: estimating velocity from neighbouring frames

The simplest estimate is the average velocity between neighbouring frames. To make the estimate belong to the time of one frame, we usually take the frames on either side. This is the three-point central difference:

The hat marks an estimate. It has two kinds of error, with different causes, and they move in opposite directions as the frame rate changes.

The first is truncation error, from replacing an instant by a finite stretch of time. For constant acceleration the formula is exact: substitute $x=p+qt+rt^2$ and the $rt^2$ parts of the two terms subtract to leave $q+2rt$, which is the instantaneous velocity. For a sinusoid, Section 2.2 already showed that the difference gives the true velocity times $\sin(\omega\Delta t)/(\omega\Delta t)$.

At $30\,\mathrm{Hz}$, $\omega\Delta t=\pi/30\approx0.105$ and the ratio is 0.99817. The velocity is underestimated by about 0.18%, an error amplitude of about $0.0086\,\mathrm{m\,s^{-1}}$ and an RMS of about $0.0061\,\mathrm{m\,s^{-1}}$. With the Lab's noise and rounding switched off, the three-point velocity RMS error is $0.00606\,\mathrm{m\,s^{-1}}$.

The second error comes from position noise. Each of the two positions has an error with standard deviation $\sigma_x$. Their difference has a standard deviation of $\sqrt2\,\sigma_x$, and dividing by $2\Delta t$ gives

With $\sigma_x=0.0034\,\mathrm m$ and $\Delta t=1/30\,\mathrm s$ this is about $0.072\,\mathrm{m\,s^{-1}}$. On the Lab's default record, the three-point difference gives a velocity RMS error of $0.068\,\mathrm{m\,s^{-1}}$. That is more than ten times the truncation error, so noise is the main problem here.

Pixels make the point clearer. The three-point difference can be written $(u_{n+1}-u_{n-1})/(2\Delta t\,s)$, and with the default settings the denominator is 8 pixel seconds per metre. A difference of a single pixel between the two frames changes the velocity estimate by $0.125\,\mathrm{m\,s^{-1}}$.

A higher frame rate reduces truncation error but amplifies noise. At $120\,\mathrm{Hz}$ the raw position error is about $0.0031\,\mathrm m$, the formula predicts about $0.27\,\mathrm{m\,s^{-1}}$, and the Lab gives $0.264\,\mathrm{m\,s^{-1}}$, roughly four times the $30\,\mathrm{Hz}$ value. Filming more often makes the three-point difference worse.

Acceleration needs a second difference, which amplifies noise even more. On the default record the three-point formula $(x_{n+1}-2x_n+x_{n-1})/\Delta t^2$ has an acceleration RMS error of about $8.0\,\mathrm{m\,s^{-2}}$, comparable with the true acceleration amplitude $A\omega^2\approx14.8\,\mathrm{m\,s^{-2}}$.

2.6 Local fits: trading more frames for smoothness

With only two frames there is nothing to average the noise over. The remedy is to take $2m+1$ frames centred on frame $n$, fit a quadratic curve through them, and use the curve's slope at the centre.

The fit chooses the curve that makes the sum of squared vertical distances from the points smallest. This is a least-squares fit, discussed in detail in Chapter 3. Savitzky and Golay in 1964 gave a convenient algorithm for smoothing and differentiating data with such local polynomial fits.

Label the frames in the window by offsets $k=-m,\dots,m$. Because the window is symmetric, $k$ and $k^3$ both sum to zero, so the constant and quadratic terms do not affect the slope. The centre slope equals the slope of a straight-line fit:

With $m=1$ this is exactly the three-point difference. As the window widens, $\sum k^2$ in the denominator grows quickly and more of the noise averages out:

A nine-frame window has $\sum k^2=60$, leaving only about $0.013\,\mathrm{m\,s^{-1}}$ of noise. The price is smoothing. Nine frames span $0.27\,\mathrm s$, during which the velocity changes appreciably, and the fitted slope flattens its peaks and troughs. Table 2.1 lists the results on the default record.

The velocity error falls and then rises again, with its minimum near five frames. The nine-frame error comes mostly from smoothing: with noise and rounding switched off, its velocity error is still $0.069\,\mathrm{m\,s^{-1}}$, almost the same as with noise. Acceleration needs a wider window to suppress the noise, so the best window differs between the two columns.

How wide a window should be depends on how long it lasts compared with how fast the motion changes, not on the number of frames alone. At $120\,\mathrm{Hz}$ a thirty-one-frame window spans only $0.25\,\mathrm s$, and its velocity RMS error is $0.052\,\mathrm{m\,s^{-1}}$, better than the nine-frame window at $30\,\mathrm{Hz}$.

2.7 Sampling too sparsely: aliasing

All the methods so far assume the motion changes little between neighbouring frames. If an oscillation is faster than the camera, the record itself misleads.

Shannon showed in 1949 that a signal containing no frequencies above $f_{\max}$ is completely determined by samples taken $1/(2f_{\max})$ apart. The other side of the coin: at frame rate $f_s$, an oscillation faster than $f_s/2$ masquerades in the record as a slower one. This is aliasing.

The Lab's “Aliased motion” preset oscillates the ball at $3\,\mathrm{Hz}$ while the camera takes only 4 frames per second. In frame $n$ the sine is

The record is identical to that of a $1\,\mathrm{Hz}$ oscillation with the opposite sign. Velocities estimated from these frames have an RMS error of $15.6\,\mathrm{m\,s^{-1}}$, while the true velocity amplitude is $18.8\,\mathrm{m\,s^{-1}}$.

The “Stroboscopic rest” preset is more extreme: a $4\,\mathrm{Hz}$ oscillation filmed at 2 frames per second. Every frame lands on $\sin(4\pi n)=0$, so the ball seems not to move at all. In fact it passes the origin at $8\pi\,\mathrm{m\,s^{-1}}\approx25.1\,\mathrm{m\,s^{-1}}$ every time a frame is taken, which is exactly the velocity error the Lab reports.

No derivative method can rescue this, because the information never entered the record. The only check is to raise the frame rate and see whether the record changes.

2.8 Integrating velocity back to position

Chapter 1 showed that position is the starting position plus the integral of velocity over time. With one velocity estimate per frame, we can average the velocities of neighbouring frames, multiply by the frame interval and add up the pieces. This is the trapezoidal rule.

In such a sum, random errors are sometimes positive and sometimes negative, so they partly cancel. A constant velocity bias keeps accumulating instead. The “Velocity-bias drift” preset adds $0.2\,\mathrm{m\,s^{-1}}$ to every velocity estimate. Integration runs from the first valid estimate to the last, $3.733\,\mathrm s$ in all, so the integrated position ends ahead of the raw position by

That is exactly the Lab's Last segment integration minus raw position reading. The default nine-frame fit, with no bias added, gives only $0.026\,\mathrm m$.

Another problem is a gap in the record. In the “Leaves the sensor” preset the ball runs out of the picture near the end, and only 114 of the 121 frames have a pixel. A nine-frame window that misses even one frame gives no estimate, so 106 velocity estimates remain. An honest estimator leaves those frames empty.

2.9 Worked example: velocity from five frames of pixels

Below are the pixel columns of frames 13 to 17 in the Lab's default record, from $0.433\,\mathrm s$ to $0.567\,\mathrm s$. Take $s=120\,\mathrm{px\,m^{-1}}$, $u_0=1024$ and $\Delta t=1/30\,\mathrm s$. Find the three-point and five-frame velocity estimates at frame 15, and decide whether the difference between them is unusual.

First convert to positions; frame 13, for example, is at $(1208-1024)/120\,\mathrm m=1.5333\,\mathrm m$. Around these frames the ball reaches its furthest point to the right and starts back.

The three-point difference uses only frames 14 and 16:

In pixels, the two frames differ by 2 pixels; divided by 8 pixel seconds per metre, that is again $0.25\,\mathrm{m\,s^{-1}}$.

The five-frame fit uses $k=-2,\dots,2$, with $\sum k^2=10$:

The two estimates differ by $0.10\,\mathrm{m\,s^{-1}}$. From Section 2.5, the three-point noise has a standard deviation of about $0.072\,\mathrm{m\,s^{-1}}$ and the five-frame fit about $0.032\,\mathrm{m\,s^{-1}}$. The difference is within what noise allows and does not mean either calculation is wrong.

Finally, compare with the truth. From the formula in Section 2.2, the cosine term is zero at $0.5\,\mathrm s$, so the true velocity is just the drift velocity, $0.15\,\mathrm{m\,s^{-1}}$. The five-frame fit happens to hit it this time, and the three-point difference is high by less than one pixel's worth. A real experiment has no such truth, and you would rely on the kind of noise estimate in the previous paragraph to judge whether a result is reasonable.

2.10 Checking in the Motion Measurement Lab

Open the Motion Measurement and Numerical Differentiation Lab. A record has already been acquired when the page opens, and the readouts show the errors over the whole record. “Export raw pixels” saves the camera record, without the truth.

Compare derivative methods. Choose “Noisy oscillation”. Predict first: if you set “Derivative: 0 three points; 1 quadratic fit” to 0, will the velocity RMS error be larger or smaller than with the default nine-frame fit? Set it back to 1 and set “Quadratic fit window in frames” to 5, 15 and 31 in turn. Note the velocity and acceleration RMS errors and compare them with Table 2.1. Then set “Pixel noise bound before rounding” to 0 and “Integer pixel rounding” to 0. Which window's error hardly changes, and why?

Change the frame rate. Still with the three-point difference, raise “Frame rate” from 30 to 120. Before you do, use the formula in Section 2.5 to predict the new velocity error.

Check the calibration. Choose “Uniform motion”, set “Assumed scale error” to 10%, and note the raw position and velocity RMS errors. Press “Reset”, choose “Uniform motion” again, and this time set “Assumed origin pixel error” to 12. Explain why one error affects the velocity and the other does not.

Check the sampling. Choose “Aliased motion” and “Stroboscopic rest”. Use Section 2.7 to predict what the camera will see, then raise the frame rate step by step. Find the frame rate at which the record starts to follow the true motion and compare it with $2f$.

Table 2.3 collects the results for these settings. The Lab's perturbation and rounding are deterministic, so the same settings always give the same numbers. Changing Deterministic noise realization swaps in a different set of perturbations: across the 8 realizations we tried, the three-point velocity RMS error ranged from 0.059 to $0.072\,\mathrm{m\,s^{-1}}$.

2.11 Exercises

1. Which quantities does a camera record give? Why are position and velocity both estimates?
2. A camera has a scale of 200 pixels per metre with the origin at column 1024. A ball appears at column 1124 and then at column 1174, in two frames $0.1\,\mathrm s$ apart. Find the average velocity over that time. Does the answer change if the origin is really at column 1030? If the scale is really 190 pixels per metre?
3. A set of errors has a mean of $0.03\,\mathrm m$ and an RMS of $0.05\,\mathrm m$. What is the standard deviation? Is the set dominated by an overall offset or by spread?
4. Position noise has a standard deviation of $2\,\mathrm{mm}$ at $60\,\mathrm{Hz}$. Roughly how large is the velocity noise of the three-point difference? Of a seven-frame fit?
5. Use the result of Section 2.2 to show that the three-point difference has no truncation error for constant acceleration, and that for a sinusoidal oscillation it underestimates the velocity more at lower frame rates. At $10\,\mathrm{Hz}$, by what percentage is the default motion's velocity underestimated?
6. A classmate says, "The higher the frame rate, the more accurate the velocity." Use this chapter's results to say when this is true and when it is false.
7. A fan turns 25 times a second. In a video at 24 frames per second, does it seem to turn slowly backwards or forwards, and how many turns per second does it seem to make?
8. Transfer. You film a cart sliding across a table with a phone at 240 frames per second. The scale is about 1000 pixels per metre and the tracking error about 1 pixel. The cart moves at about $0.5\,\mathrm{m\,s^{-1}}$ and its velocity changes little over $0.2\,\mathrm s$. How would you choose the derivative method and the window? Give the expected velocity noise and say how you would check that your choice is suitable.

2.12 Answers and explanations

Next: testing a model with one fall

This chapter estimated velocity from a record without assuming any law for the motion. The next chapter takes a different route. It assumes the acceleration is constant, derives how position depends on time, tests that assumption against the whole record, and measures the acceleration of gravity.

Figure and data notes

• The derivative of the sine and the error of the three-point difference are derived independently in this chapter. The definitions of average and instantaneous velocity follow OpenStax University Physics Volume 1, Section 3.2.
• Tables 2.1, 2.2 and 2.3 and Figure 2.1 were computed with the Motion Measurement Lab's model code at the Lab defaults: $4\,\mathrm s$ duration, $30\,\mathrm{Hz}$ frame rate, $120\,\mathrm{px\,m^{-1}}$ scale, pixel noise bound 0.5 pixels with rounding, deterministic noise realization 17. Other settings in the tables change only the parameters named. The reference velocity comes from the Lab's motion formula and is used only for comparison.
• The noise estimates in the text treat the Lab's deterministic perturbation as an even spread, to check orders of magnitude. They are not an exact description of the Lab's noise.