Median vs mean filter, interactive demo

Two kinds of noise

Gaussian noise adds a small random value to every pixel, $I = I_0 + \eta$ with $\eta \sim \mathcal{N}(0, \sigma^2)$ , like the electronic noise of a sensor.
Salt-and-pepper noise leaves most pixels intact but sets a fraction of them to black or white, like dead pixels or transmission errors. These outliers have nothing to do with the true value.

Mean filter

The box filter, or mean filter, replaces each pixel by the average of the $N$ pixel values $I_i$ in its window, here $N = n \times n$ :

B = \frac{1}{N} \sum_{i=1}^{N} I_i

Averaging $N$ independent noise values reduces their standard deviation by a factor $\sqrt{N} = n$ , which works well for Gaussian noise. But the mean is sensitive to outliers: a single white pixel on black contributes $1/N$ to every output pixel of its window and leaves a gray square. The mean also mixes the two sides of every edge, so edges get blurred.

Median filter

The median filter selects the median intensity in the window: it sorts the values and takes the middle one. Because it is based on the ordering of the gray levels, it is a rank filter, like the min, max and range filters.

M = \operatorname{median} \{ I_1, \dots, I_N \}

As long as fewer than half of the values in the window are outliers, they end up at the ends of the sorted list and do not affect the result at all. At a straight edge, more than half of the window lies on one side, so the output takes a value from that side: the edge stays sharp. The price is that thin lines and corners, which cover less than half of the window, are removed, and sorting makes it more expensive than the mean.

The median is not a weighted sum of the inputs, so it is a non-linear filter: it cannot be written as a convolution, and the median of a sum of two images is not the sum of their medians.

Measuring the result. The values panel compares every image with the original using the peak signal-to-noise ratio

\mathrm{PSNR} = 10 \log_{10} (1 / \mathrm{MSE})

dB. Higher is better.

Try this

With salt-and-pepper noise, compare the two filters at size 3: the median removes nearly all dots, the mean turns each into a gray blotch.
Select a noisy pixel and look at the sorted window in the values panel: the black and white values sit at the ends, far from the median.
Raise the density to 0.3: size 3 is no longer enough for the median, because windows now often contain many outliers. Try size 5.
Switch to Gaussian noise: now the mean holds up much better, and the PSNR values of the two filters are close.
Follow an edge in the row profile: the median follows the step, the mean turns it into a ramp as wide as the window.
Choose the test chart and filter size 7: the median rounds off corners and removes the finest lines completely.