Canny edge detector

What makes a good edge detector

• Good detection: it finds all real edges and ignores noise and other artifacts.
• Good localization: the detected edges are as close as possible to the true edges, and the detector returns only one point for each true edge point.

Canny modeled edges as ideal steps corrupted by additive Gaussian noise and looked for the linear filter that optimizes the product of the signal-to-noise ratio and the localization. The first derivative of a Gaussian approximates this optimal operator closely. Filtering alone still gives thick, broken responses, so three more steps turn them into thin, connected curves.

Step 1: derivatives of Gaussian

The image is filtered with the $x$ and $y$ derivatives of a Gaussian, which smooths and differentiates at the same time, as explained in Derivatives & noise:

$I_x$ responds to vertical edges, $I_y$ to horizontal ones. Positive values (bright) mean the intensity increases to the right or downwards.

Step 2: magnitude and orientation

Together, the two derivatives form the gradient $\nabla I = (I_x, I_y)$. It points in the direction of the steepest increase in intensity, across the edge, and its length measures the edge strength:

Because $y$ points down in images, $\theta$ turns clockwise on the screen. The two sides of a thin line have opposite gradients, so they show up in opposite colors in the orientation view.

Step 3: non-maximum suppression

The magnitude forms ridges that are several pixels wide, and wider for larger $\sigma$. To get one point per edge, a pixel $q$ is kept only if its magnitude is a maximum along the gradient direction. It is compared with the points one pixel ahead and one pixel behind,

which usually fall between pixel centers, so their magnitudes are interpolated from the four surrounding pixels. The pixel survives if $\|\nabla I\|(q) > \|\nabla I\|(p)$ and $\|\nabla I\|(q) \ge \|\nabla I\|(r)$; the asymmetric comparison keeps exactly one pixel of a ridge with two equal values. Select a pixel and zoom in to see $p$ and $r$ drawn on the magnitude views.

Step 4: hysteresis thresholding

The thinned magnitude still contains small responses from noise and texture. A single threshold either lets them through or breaks real edges into dashes wherever their contrast dips. Hysteresis uses two thresholds:

• magnitude above the high threshold: strong edge pixel,
• magnitude below the low threshold: noise, discarded,
• magnitude in between: weak edge pixel.

Starting from the strong pixels, edges are followed into connected weak pixels, which is a connected components search over the 8 neighbors of each pixel. A curve has to be strong in one place only, and it is then kept as long as it stays above the low threshold. Here both thresholds are given as fractions of the largest magnitude in the image.

Choosing σ

The choice of $\sigma$ depends on the desired behavior: a large $\sigma$ detects large-scale edges and suppresses noise, a small $\sigma$ detects fine features. Canny finds every intensity edge at the chosen scale, including texture and shadow boundaries. Which of them are useful is left to the application, which can only choose if all edges were found in the first place.

Try this

• Select a pixel on an edge and zoom to 8×: the magnitude ridge is several pixels wide, and only its center survives non-maximum suppression.
• Set both thresholds to the same value, so that there is no hysteresis. Lower it until the edges are closed, and look at how much noise comes with them.
• Now raise only the high threshold: noise disappears, while real edges stay connected through their weak parts.
• Add noise with $\sigma_n = 0.05$ and compare $\sigma = 0.8$ with $\sigma = 3$: small scales find noise and fine texture, large scales only the main outlines.
• Choose the test chart and increase $\sigma$: the finest line groups vanish first, and corners get rounded.
• Look at the orientation view of the colored text: letters on dark and on bright backgrounds have opposite colors, because their gradients point the other way.