Dither

Take a smooth gradient from dark to light, 256 pixels wide, and store it with only 8 shades. Watch hard bands appear where the shade jumps, and then watch them dissolve into fine grain.

Bands in a smooth sky

Quantization and noise (11.1) ended on a quiet sine. The error of a quiet or simple signal repeats with the signal, so it does not sound like hiss. It sounds like extra harmonics, a distortion that follows the music. This page is about a cure that sounds strange at first: add a little noise before you round.

I started with a picture, because a picture shows the problem fastest. In What a signal is (1.1) you saw that one row of a picture is a signal: brightness at each pixel position. Store a smooth gradient with only 8 shades and the rounding error is large, and it is not random. It is the same small sawtooth each time the brightness crosses a level, so your eye sees it as banding: hard edges where the shade jumps.

In the last step of the picture at the top I add dither: a small random value, spread evenly over plus or minus half a shade, added to each pixel before rounding. The picture still has only 8 shades, but the bands dissolve into fine grain.

Look at the strip of brightness against position under the picture at the top. With dither, the rounded values flicker between neighbouring shades instead of switching once. That flicker is what your eye averages into a smooth gradient. A newspaper photo works the same way: each dot is black or white, and the grey you see is the average over a small area.

Right on average

The picture suggests an idea about averages, and I want you to see it with numbers. Dither adds noise, so you would expect the output to get worse. Before you run the next instrument, decide: I will add noise to a quiet signal before rounding it. Will the result get better or worse?

Here is the setup. The input is a sine 1.4 steps tall, with 32 samples per period. A step is the distance $\Delta$ between neighbouring levels, and the levels sit at whole steps: $\ldots,-1,0,1,\ldots$ Without dither, the quantizer $Q\{\cdot\}$ rounds each sample to the nearest level. Every period gives the same output, a fixed staircase.

With dither, we round $x[n]+v[n]$ instead, where $v[n]$ is a fresh random value for each sample, spread evenly between $-\tfrac12$ and $+\tfrac12$ step. This is rectangular dither. The question is what one run does to a sample, and what many runs do on average.

With one run, the gaps read 0.464 without dither and 0.778 with it, so dither is worse for a single run. Watch the stems as the count rises. At 10 runs the gap with dither has fallen to 0.273, and at 100 runs to 0.102. Without dither the stems never move: every run is the same, so the gap stays at 0.464.

Here is why. Suppose the input sits a share $p$ of the way from one level to the next, so it is $p$ steps above the lower level, with $0\le p<1$. The sum $x+v$ spreads over $\pm\tfrac12$ step around the input. The output rounds up when $v\ge\tfrac12-p$, a stretch of values of length $p$ out of a total length 1, so it rounds up in a share $p$ of the runs. So the output is the upper level in a share $p$ of the runs and the lower level in the rest.

Average those outputs and you get $p$ steps above the lower level, which is the input. The average of many runs closes in on the input. That is what “right on average” means.

Press Hear without dither and Hear with dither. Each plays that panel’s output for a single run, for 2 seconds: a 1.5 kHz tone at 48 kHz, with a step of 0.05 of full scale. The second has a steady hiss. I think of it this way: the error without dither repeats every period, so it is made of harmonics (see Signals as sums of sinusoids, 7.1), and the third harmonic of the error is 0.273 of a step. The error with dither never repeats, so it is hiss, which the ear tolerates far better than a tone that follows the music.

Dither has a cost. The rounding error’s mean square is $\Delta^2/12$, as in 11.1. The dither adds its own mean square, which for a value spread evenly over $\pm\tfrac12$ step is also $\Delta^2/12$. Averaged over the input position, the total is $\Delta^2/6$: twice as much, which is $10\log_{10}2=3.01$ dB more noise.

Rectangular or triangular dither

Rectangular dither fixes the average, but one problem remains. How much hiss you get still depends on where the input sits between two levels. Put the input exactly on a level and the dither never pushes it across the halfway point, so the output is exact and there is no error. Put it halfway between two levels and the error is as large as it gets.

Here is the mean square of the error for an input $p$ steps above a level. The output is up by $1-p$ steps from the input with probability $p$, and down by $p$ steps with probability $1-p$, so

It is zero at $p=0$ and at $p=1$, and largest at $p=\tfrac12$, where it is $0.25\,\Delta^2$.

When the input is a slow fade, the hiss swells and shrinks with it. That is noise modulation, hiss that breathes with the music. A fading piano note is the classic case: the background hiss pulses along with the note.

The fix is triangular dither: the sum of two independent rectangular values. It spreads over $\pm1$ step and is most often near 0. For this shape the mean square of the error is $\Delta^2/4$ for every input. Watch both kinds of dither as a steady input moves from one level to the next.

The rectangular curve is an arch from 0 up to 0.25 and back, and the triangular line stays flat at 0.25. A steady hiss does not breathe, and that is the reason triangular dither is the usual choice.

Its cost is $\Delta^2/4$ against the undithered $\Delta^2/12$: three times the noise, or $10\log_{10}3=4.77$ dB. Compare rectangular dither’s 3.01 dB.

There is one way to avoid most of that cost. In subtractive dither (Roberts’ method, in MIT 6.003 lecture 22) you add known noise, round, and subtract the same noise afterwards. The error is then $\Delta^2/12$ and does not depend on the input. But both ends must share the noise, so most audio uses triangular dither instead. When a 24-bit master is reduced to 16 bits, triangular dither is the usual choice.

Worked example

1. No dither. Take an input $0.3$ steps above a level. It always rounds down, so the error is $-0.3$ steps every time, and the mean square is $0.3^2=0.09$ in units of $\Delta^2$.
2. Rectangular dither. The output is the upper level in a share $0.3$ of the runs, so the average output is $0.3$ steps above the lower level, the input. The mean square of the error is $p(1-p)=0.3\times0.7=0.21$.
3. Triangular dither. The mean square is $0.25$ here, as for every input. The rectangular value moves with the input, and the triangular one does not.
4. Averaged over the input. Undithered, $\tfrac1{12}=0.0833$. Rectangular, $\tfrac16=0.1667$, a factor 2, so $+3.01$ dB. Triangular, $\tfrac14=0.25$, a factor 3, so $+4.77$ dB.

Where you’ll meet this

Every time audio is reduced to fewer bits, dither is applied first, usually triangular. The same idea prints photographs with a few ink colours, and it keeps a slowly fading screen gradient from showing bands.

Dither makes the error steady, but it does not make it smaller. The next page, Oversampling and noise shaping (11.3), shows how to move that hiss to frequencies where it matters less.

Reference card