Derivatives & noise for edge detection, interactive demo

Where edges come from

Edges are discontinuities in an image, and they are caused by discontinuities in the scene:

of the surface normal, where the orientation of a surface changes, like the fold of a box,
of depth, where one object ends in front of another,
of the surface color, like printed text or the texture of a material,
of the illumination, like the border of a shadow.

Edges are much more compact than the full image and still carry most of its structure, so they help to recover the geometry of a scene and the viewpoint, and they are a basis for recognition.

Edges are peaks of the derivative

Take a single row of the image: the intensity as a function of position is a 1D signal $f(x)$ . An edge is a place where $f$ changes rapidly, so it corresponds to an extremum of the first derivative and to a zero crossing of the second derivative. On pixels the derivative is approximated by a difference; this demo uses the central difference

\frac{d f}{dx}[x] \approx \frac{f[x + 1] - f[x - 1]}{2},

which is a linear filter with the weights $[-\tfrac{1}{2} \;\; 0 \;\; \tfrac{1}{2}]$ applied by correlation. A dark-to-bright step is positive, a bright-to-dark step negative. The Sobel filter $S_x$ uses the same central difference along $x$ and adds a small smoothing $[1 \;\; 2 \;\; 1]$ along $y$ .

Difference filters respond strongly to noise

Image noise makes pixels look different from their neighbors, and a difference filter measures exactly that. The larger the noise, the stronger the response. For independent noise with standard deviation $\sigma_n$ in every pixel, the central difference has noise with standard deviation $\sigma_n / \sqrt{2}$ , no matter how gentle the real edges are. A sharp step of height $h$ still stands out with a peak of $h / 2$ , but a blurry edge spreads the same change over many pixels, and its small derivative disappears in the noise. Thresholding the derivative then finds edges everywhere.

Smooth first

The solution is to smooth the signal with a filter $g$ , such as a Gaussian, and look for peaks in $\frac{d}{dx}(f * g)$ . Differentiation is itself a convolution, and convolution is associative, so smoothing and differentiating can be combined into one filter, the derivative of the Gaussian:

\frac{d}{dx}(f * g) = f * \frac{d g}{dx}, \qquad G'_\sigma(x) = -\frac{x}{\sigma^2} \, G_\sigma(x) = -\frac{x}{\sqrt{2\pi}\,\sigma^3} \, e^{-\frac{x^2}{2\sigma^2}}

This saves one operation. In the demo, $\frac{dg}{dx}$ is the central difference of the sampled Gaussian, so the two curves in the derivative plot agree up to rounding, and it follows the continuous $G'_\sigma$ closely. For an image, the Gaussian derivative in one direction is a 2D kernel that differentiates along $x$ and smooths along $y$ :

\frac{\partial G_\sigma}{\partial x}(x, y) = G'_\sigma(x) \, G_\sigma(y)

With “smooth along x and y”, the rows above and below the selected one are averaged in as well, which removes much more noise than smoothing along the row alone.

Choosing σ

A larger $\sigma$ removes more noise, but it also spreads each edge: the peak gets lower and wider, which makes its position less precise, and nearby edges merge into one. Small $\sigma$ finds fine details, large $\sigma$ only the large-scale edges. Detecting all real edges and localizing them precisely pull in opposite directions; the Canny edge detector is built around this trade-off.

The second derivative

Every extremum of the first derivative is a zero crossing of the second, so edges can be found as zero crossings, too. But in a flat, noisy region the second derivative crosses zero all the time. The demo only marks crossings where the first derivative reaches the threshold $t$ , and these coincide with the peaks. Differentiating twice amplifies noise even more, so the second derivative needs smoothing all the more.

Try this

Set the noise to 0 and follow the selected row: every edge is a peak in the first derivative and a zero crossing in the second.
Raise the noise to 0.05 with the smallest σ: the raw derivative image is covered in noise, and the values panel counts dozens of peaks above the threshold.
Now increase σ until only the real edges remain as peaks. Watch the peaks get lower and wider at the same time.
Compare the accent and the dashed curve in the derivative plot: smoothing and then differentiating is the same as filtering once with the derivative of Gaussian.
Switch between smoothing along the row and along x and y: with the same σ, the 2D version removes much more noise from the derivative.
Turn on “edges of all rows” to search every row on its own. The raw derivative view marks the peaks without smoothing, and with noise they are everywhere; after smoothing they line up along the outlines. Horizontal edges are missing, because a row only sees changes along x. The Canny edge detector uses both derivatives.
Choose the test chart and a large σ: the closely spaced lines merge, and their edges disappear from the derivative.