Oversampling and noise shaping

The same quantizer error, shared among more slices of frequency. Watch the noise in the band fall while the total stays put.

Sharing the same error among more slices

Quantization & noise (11.1) fixed the quantizer’s error power at $\Delta^2/12$ per sample, where $\Delta$ is the step. That number depends on the step and on nothing else, so sampling faster does not change it. There are two ways to a cleaner signal from the same bits. This page takes both: sample faster and filter, then push the error up in frequency with feedback.

Here is the idea behind that picture. For a busy signal the error behaves like hiss, and its power $\Delta^2/12$ is spread evenly over all frequencies from 0 to $f_s/2$. I state this here; 24.4 proves it. Your signal occupies only the band from 0 to $f_B$. Cut the range 0 to $f_s/2$ into slices, each $f_B$ wide. The number of slices is the oversampling ratio,

At $\mathrm{OSR}=1$ there is one slice: you sample at $2f_B$, the slowest rate that still holds the band (The sampling theorem, 10.2).

The picture at the top of the page starts there. The noise is in decibels relative to $\mathrm{OSR}=1$, the power ratio $10\log_{10}$ from How big is a signal (1.3). Sampling at exactly twice the band, all of the error’s power, $\Delta^2/12$, sits in the band.

At $\mathrm{OSR}=2$, twice as fast, the same power is shared by two slices. The filter keeps one: −3.01 dB. At $\mathrm{OSR}=4$ the same power is shared by four slices, and a digital low-pass that keeps only the first one keeps a quarter of it. That is the ideal low-pass of Frequency response and Bode plots (8.4), done in software. A quarter of the power is

Each doubling of the rate, one octave, halves the power that is kept: $-3.01$ dB, because $10\log_{10}2=3.01$. At $\mathrm{OSR}=8$, eight times faster, there are eight slices, and the filter keeps one: −9.03 dB. At $\mathrm{OSR}=16$ the band holds $-12.04$ dB. At 6.02 dB per bit that is two extra bits of quality.

Look at the total noise. It never leaves 0.00 dB. Sampling faster only spreads the error thinner. Without the low-pass nothing is gained; the filter, keeping the first slice, does all the work.

Averaging four readings of a noisy scale halves the noise’s RMS. That is the same arithmetic: four readings, a quarter of the power.

What a first difference does to slow and fast wiggles

Now the second move. Draw the loop as in Difference equations (6.1): quantize, keep the quantizer’s error in a delay box, and subtract it from the next input. The quantizer adds its own error $e[n]$ to whatever enters it, so with the stored error taken off the input,

This is error feedback. The error reaching the output is $e[n]-{e[n-1]}$, which is the first difference of $e$ from Operations on amplitude (2.2).

How does a first difference treat slow and fast wiggles? I measure it with test sines, as in Properties of LTI systems (5.4): a sine goes in, a sine of the same frequency comes out, scaled. The scale is what I want. The next picture runs four test sines, from slow to almost the fastest wiggle, $\Omega=\pi$ (Sinusoids, 3.2). Watch the first difference shrink the slow one and grow the fast one.

The slow test sine, $\Omega=\pi/20$, has a first difference only 0.157 of its size. At $\Omega=\pi/3$, one cycle every 6 samples, the first difference is exactly as big as the input, 1.000. At $\Omega=\pi/2$, one cycle every 4 samples, it is 1.414 times the input. At $\Omega=19\pi/20$, almost the fastest wiggle there is, it is 1.994 times the input.

The dots rise from almost 0 to almost 2. At $\Omega=\pi/3$ the ratio is 1.000, the crossover: slower wiggles shrink, faster ones grow. A first difference nearly cancels slow wiggles and doubles the fastest ones. Subtracting yesterday’s temperature from today’s does the same: the slow seasonal drift nearly vanishes, and the day-to-day jumps stay.

The dots lie on a smooth curve from 0 to 2. Its formula comes with the DTFT, in Frequency response of discrete-time systems (12.4). Power goes as size squared (1.3), so shaping multiplies the error’s power at $\Omega$ by the square of the dot’s value: 0.025 at $\pi/20$ and 3.98 at $19\pi/20$. Feed the error back twice and the error is differenced twice. That is second order shaping, and it squares the sizes again: 0.025, 1.000, 2.000 and 3.975 at the four tests.

Push the error up, then filter it away

Put the two moves together at $\mathrm{OSR}=16$. Shaping multiplies each slice’s power by the curve squared. The band’s slice nearly empties, and the slices at the top fill up. I can then low-pass, keep the first slice, and throw the rest away. Watch the band’s bar empty as the top bars grow.

With no shaping every bar is $0.0625$, a sixteenth. First-order bars run from 0.0008 and 0.0056 in the lowest slices to 0.2492 in the top one. Second-order bars run from 0.00002 and 0.0006 to 0.9936. Feed the error back once, and its power moves toward the top slices. The band’s slice drops to −30.96 dB, though the total rose by 3.01 dB. Feed it back twice, and the band holds −47.33 dB while the total is +7.78 dB.

Notice that the noise in the band falls while the total noise rises. Adding the sixteen bars gives 2 for first order and 6 for second, so the total power is multiplied by 2 and by 6, which is $+3.01$ dB and $+7.78$ dB. Shaping moves the error and also adds to it. It still wins, because the part it adds sits in slices that the low-pass removes.

That is sweeping dust into the corner you will vacuum. Left out of this page: stability for orders above 2, cascaded loops, and the design of the decimation filter, which waits for Chapter 22.

One bit at a time

Take a first-order loop whose quantizer has only one bit, so its output is $+1$ or $-1$, and give it a constant input of 0.5. The bits run $1,1,-1,1$ and repeat. Their running average is 0.5. With an input of 0.25 the bits are $1,-1,1,1,-1,1,-1,1,\dots$ and their average over 16 bits is 0.25.

The average over many bits is the digital low-pass. A 1-bit loop followed by that average is a sigma-delta converter. Such loops can settle into short repeating patterns, called idle tones, which Dither (11.2) breaks up.

Worked example

Every number below was computed with $\mathrm{OSR}=f_s/(2f_B)$ and the 6.02 dB per bit rule.

1. Oversampling alone. In-band noise relative to $\mathrm{OSR}=1$ for $\mathrm{OSR}=2,4,8,16,32,64$ is $-3.01$, $-6.02$, $-9.03$, $-12.04$, $-15.05$, $-18.06$ dB. That is 3.01 dB per octave. $\mathrm{OSR}=16$ is worth 2.0 bits, and $\mathrm{OSR}=64$ is worth 3.0 bits.
2. First-order shaping. At $\mathrm{OSR}=16$ the band holds $-30.96$ dB, or 5.1 bits. At $\mathrm{OSR}=64$ it holds $-49.01$ dB, or 8.1 bits. Between those two the gain is 9.03 dB per octave.
3. Second-order shaping. At $\mathrm{OSR}=16$ the band holds $-47.33$ dB, or 7.9 bits. At $\mathrm{OSR}=64$ it holds $-77.41$ dB, or 12.9 bits. The gain is 15.05 dB per octave.
4. Totals. Shaped error power is multiplied by 2, 6 and 20 for orders 1, 2 and 3: $+3.01$, $+7.78$ and $+13.01$ dB.
5. Test sines. Size out ÷ size in through one first difference, at $\Omega=\pi/20,\ \pi/3,\ \pi/2,\ 19\pi/20$: 0.157, 1.000, 1.414, 1.994. Through two: 0.025, 1.000, 2.000, 3.975.
6. A one-bit stream. Input 0.5 gives $1,1,-1,1$ repeating, average 0.5. Input 0.25 gives an average of 0.25 over 16 bits.

Where you’ll meet this

The curve you measured with test sines gets its formula in the DTFT, in 12.4. Cutting the sample rate after the low-pass, called decimation, is the subject of Chapter 22. Audio converters use these loops, and Chapter 29 comes back to them.

Reference card