Convergence and the Gibbs phenomenon

Will the bump at the corner of a square wave shrink to nothing as arrows are added? Decide, then watch.

Stopping after a few arrows

On the page Signals as sums of sinusoids (7.1) the square wave was built from spinning arrows, and every arrow that joined made the curve closer to the target. I also said that the little spikes at the corners do not go away. This page says exactly what happens when you stop after a given number of arrows, what shrinks, what refuses to, and why.

The wave is 7.1’s square wave between $-1$ and $+1$. Its arrows are the odd harmonics $k=1,3,5,\dots$, with lengths $b_k=4/(\pi k)$. I call the sum of the arrows you keep a partial sum. I count arrows in words, “ten arrows”, and I name a partial sum by the highest harmonic it keeps, $K$. For the square wave, ten arrows reach harmonic 19, so $K=19$. In general $K$ is twice the number of arrows, minus 1.

Go back to the picture at the top of the page. With two arrows the tallest point is $1.200$, at $\theta=\pi/4$. With ten it is $1.180$, at $\theta=\pi/20$, and with twenty it is still $1.179$. At the jump itself the curve is exactly 0. So the answer to my question is no. The bump moves towards the jump and gets narrower, but its top stays near $1.18$.

The value at the jump comes from 7.1. Each arrow’s height is $b_k\sin k\theta$, and every $\sin k\theta$ is 0 at $\theta=0$. So every partial sum is exactly 0 there, the middle of the jump from $-1$ to $+1$. This holds for all arrow counts, including infinitely many. At a jump the series converges to the midpoint of the jump, and the bump sits next to it, not on it.

An old audio file that was compressed hard shows this as “ringing”: a ripple that follows every sharp click.

The bump keeps its height

Why does the bump narrow? Its top is at $\theta\approx\pi/(K+1)$, so doubling the arrows roughly halves how far from the jump it sits. Its width follows the same rule. Suppose I zoom the horizontal axis in step with the number of arrows. Then the bump should look the same every time, because the narrowing is exactly undone by the zoom.

The picture below does this, like a stronger and stronger magnifying glass on the same crack. Watch the top of the bump against the dotted ruler as the count climbs from 5 to 80 arrows.

The top of the bump stays on the ruler. A 16-fold increase in arrows has not lowered the bump at all. The reason is that near the jump the partial sum is, to a good approximation, one fixed curve that depends only on $(K+1)\theta$. More arrows do not change the curve. They only squeeze it. I do not derive that curve here. Its top is a known constant, $\tfrac2\pi\mathrm{Si}(\pi)=1.17898$, defined on the reference card, and this is the limit of the bump’s height as $K\to\infty$.

The overshoot is $0.179$ above the top at 1. It is $8.95\%$ of the jump, which is 2 high. This is the Gibbs phenomenon. Be careful with the percentage: it is $17.9\%$ of the height of the top, but about $9\%$ of the jump, and $9\%$ of the jump is the number that does not depend on how the wave is scaled.

A taller bump, a smaller error

If the bump never shrinks, does the partial sum ever become a good approximation? Yes, in a different sense, and the difference is the point of this page. Measure the error as 7.2’s power, the average of a square.

At each instant take the gap between the target $x$ and the partial sum $S_K$, square it, and average over one period. Call that the mean-square error:

The integral is an area, so $\varepsilon_K$ is the area under the squared gap divided by $T$, as on the page How big is a signal (1.3). It is also what is left in the power tank of the page Fourier series coefficients (7.2). The gap contains exactly the arrows you dropped, and by Parseval’s relation their powers add. For the square wave, the total power is 1, because $x^2=1$, and harmonic $k$ carries $8/(\pi^2k^2)$. So

For one to five arrows ($K=1,3,5,7,9$) this gives $0.1894$, $0.0994$, $0.0669$, $0.0504$ and $0.0404$.

The bump is tall but thin. It is like a thin spike on a long road: it is high, but it hardly affects the road. As the bump narrows in step with $1/K$, the area under its squared gap narrows too, even though its height stays the same.

Watch two curves grow as arrows are added: the height of the bump, and the error, the average of the shaded squared gap.

The tallest point stays flat near $1.18$, and the error keeps falling. Notice that one curve is flat and the other falls. Doubling the arrows halves the error. For ten arrows $\varepsilon_K=0.02025$, for twenty $0.01013$, and for forty $0.00507$. You can see why from the sum: the terms dropped are the odd $k>K$, and they are close together, so their sum is about half the area under $8/(\pi^2k^2)$ from $K+1$ on. That area is $\tfrac{8}{\pi^2}\cdot\tfrac{1}{K+1}$, so $\varepsilon_K\approx\dfrac{4}{\pi^2(K+1)}$. With $K+1$ equal to twice the number of arrows, this is $0.2026$ divided by the number of arrows.

The error goes to zero, while the height of the bump does not. These are two different statements about “getting closer”. The error tells you that the power in the gap vanishes. The bump tells you that the curve does not settle to the target near the jump.

Smoother waves need fewer arrows

The square wave needs many arrows because its arrows shrink slowly: arrow $k$ has length proportional to $1/k$. The cause is the jump. The page Signals as sums of sinusoids (7.1) also showed the triangle wave, which has sharp corners but no jumps, and whose arrows are $8/(\pi^2k^2)$ long. They fall as $1/k^2$.

To compare the two I measure each arrow relative to arrow 1, in decibels, as on the page How big is a signal (1.3): a ratio of lengths $r$ is $20\log_{10}r$ dB. Squaring a ratio doubles its dB value. So the triangle’s bars, which are the square of the square wave’s ratios, sit exactly twice as far down.

Watch the bars arrive in pairs, square and triangle, and compare how far down each one sits.

At $k=15$ the two lengths are $0.0667$ and $0.00444$, which is $-23.5$ and $-47.0$ dB. At $k=31$ they are $-29.8$ and $-59.7$ dB. Notice that at every $k$ the triangle’s bar is twice as far down in dB as the square wave’s.

This is also why the triangle wave sounds mellower than the square wave: its high harmonics are far quieter. A wave that is smoother still has arrows that fall faster still. The page Properties of the Fourier transform (8.2) explains why.

Which waves the series works for

So which waves can you rebuild from arrows? In one sentence: any repeating wave you can draw with a pen, with finitely many jumps and finitely many peaks and dips in each period. The reference card lists these as the Dirichlet conditions. For such a wave the arrows add up to the wave at every point where it is continuous, and to the midpoint of the jump at each jump.

More weakly, the mean-square error $\varepsilon_K$ goes to zero for any wave with finite power. That is the statement the power tank of 7.2 supports. It does not promise that the curve settles at every point, which the bump of this page has just shown.

Worked example

1. Bump height against the number of arrows (square wave between $-1$ and $+1$; the first bump’s top is at $\theta=\pi/(K+1)$). One arrow, $K=1$: $1.2732$, but this is the edge of the window, not a bump. Two arrows: $\tfrac4\pi\left(\sin\tfrac\pi4+\tfrac13\sin\tfrac{3\pi}4\right)=1.2004$. Then $1.1884$ for three, $1.1842$ for four and $1.1823$ for five. Ten arrows give $1.1798$, twenty $1.1792$ and eighty $1.1790$. The limit is $\tfrac2\pi\mathrm{Si}(\pi)=1.17898$.
2. The overshoot. $1.17898-1=0.17898$. That is $8.95\%$ of the jump of 2, and $17.9\%$ of the height of the top.
3. Error power. $\varepsilon_K=1-\sum_{k\ \text{odd}\le K}\dfrac{8}{\pi^2k^2}$. For 1 to 5 arrows: $0.18943$, $0.09937$, $0.06694$, $0.05040$, $0.04040$. For 10, 20, 40 and 80 arrows: $0.02025$, $0.01013$, $0.00507$, $0.00253$.
4. The value at the jump. Every term is $b_k\sin(k\cdot0)=0$, so every partial sum is $0$ at $\theta=0$. That is the average of $-1$ and $+1$.
5. Decay rates relative to arrow 1. At $k=15$ the square wave is $1/15=0.06667$ ($-23.52$ dB) and the triangle $1/225=0.004444$ ($-47.04$ dB). At $k=31$: $1/31=0.03226$ ($-29.83$ dB) and $1/961=0.001041$ ($-59.65$ dB). The triangle’s dB is twice the square wave’s at every $k$.

Where you’ll meet this

The ringing next to a sharp edge appears wherever you cut off the high harmonics of something with an edge. It is the ripple you see in a heavily compressed image, next to a sharp line or letter, and the ripple in the output of a sharp low-pass filter given a step. The page Window-method FIR design (19.1) meets the same ringing in filter design.

There is a remedy: do not cut the arrows off suddenly, but taper them. That is a window function, the subject of Window functions compared (15.2). The next page is Fourier series and LTI systems (7.4), where each $c_k$ passes through a system on its own.

Reference card