Pinhole camera model: intrinsic & extrinsic matrix

The pinhole model

In an ideal pinhole camera every ray of light passes through a single point, the camera center $C$ . A scene point, the camera center and its image lie on one line (the projection rays in the 3D view). The sensor sits at distance $f$ behind the pinhole, the focal length. By similar triangles, a point at camera coordinates $(X_c, Y_c, Z_c)$ lands on the sensor at

U = -f \, \frac{X_c}{Z_c}, \qquad V = -f \, \frac{Y_c}{Z_c}

The minus signs mean the image is upside down. The division by the depth $Z_c$ makes far objects look small, and it loses information: every point on a projection ray lands on the same pixel.

The 3D view shows the equivalent virtual image plane at distance $f$ in front of $C$ instead. There the image is upright and the minus signs disappear, which is the form used from here on. The plane is drawn at $Z_c = f_x / 500$ , so 1 world unit corresponds to 500 px.

The projection matrix

Division by depth is not linear, but in homogeneous coordinates the whole projection becomes one matrix product. The division only happens at the very end:

\mathbf{x} = K \, [R \mid t] \, \mathbf{X}

w \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = \begin{bmatrix} f_x & s & u_0 \\ 0 & f_y & v_0 \\ 0 & 0 & 1 \end{bmatrix} \left[\begin{array}{ccc:c} r_{11} & r_{12} & r_{13} & t_x \\ r_{21} & r_{22} & r_{23} & t_y \\ r_{31} & r_{32} & r_{33} & t_z \end{array}\right] \begin{bmatrix} X \\ Y \\ Z \\ 1 \end{bmatrix}

$P = K [R \mid t]$ is the $3 \times 4$ projection matrix. It is also often called $M$ . The pixel $(u, v)$ is found by dividing by $w$ , which here equals the depth $Z_c$ . Scaling $P$ by any non-zero factor gives the same pixels, so of its 12 entries only 11 are free. This matches the camera's 11 degrees of freedom: 5 in $K$ ( $f_x, f_y, s, u_0, v_0$ ), 3 for the rotation and 3 for the translation.

Intrinsics $K$

Start with the simplest camera: no rotation ( $R = I$ ), center at the origin ( $t = 0$ ), square pixels ( $f_x = f_y = f$ ), principal point at $(0, 0)$ and no skew:

w \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = \begin{bmatrix} f & 0 & 0 & 0 \\ 0 & f & 0 & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} X \\ Y \\ Z \\ 1 \end{bmatrix}

Real cameras relax these assumptions one at a time:

$u_0, v_0$ : the principal point, where the optical axis hits the sensor. The pixel origin is the top-left corner, so it is usually close to, but not exactly at, the image center (dashed cross). Changing it shifts the image in $u$ and $v$ .
$f_x, f_y$ : the focal length in pixels (focal length in mm divided by the pixel size). They differ only when the pixels are not square (aspect ratio $\neq 1$ ).
$s$ : skew between the pixel axes. It is essentially zero for modern sensors, but try it to see the frustum shear.

The focal length sets the angular field of view. For a sensor of size $H$ :

\mathrm{AFOV} = 2 \arctan \frac{H}{2f}

$H$ and $f$ must be in the same units. In pixels, the horizontal AFOV uses the image width (640 px) with $f_x$ , and the vertical AFOV uses the height (480 px) with $f_y$ . A shorter focal length gives a wider field of view.

Extrinsics $[R \mid t]$

The extrinsics describe the transformation from world to camera: $\mathbf{X}_c = R\,\mathbf{X}_w + t$ . The point is rotated first and then translated, so $t$ is not the camera position. It is the world origin expressed in camera coordinates. The camera center is the point that maps to $\mathbf{X}_c = 0$ :

C = -R^{-1} t = -R^\top t \qquad \text{(a rotation satisfies } R^{-1} = R^\top \text{)}

The rows of $R$ are the camera's $x$ , $y$ and $z$ axes expressed in world coordinates (the colored axes at $C$ ).

Any 3D rotation can be built from counter-clockwise rotations about the coordinate axes, for example

R_x(\alpha) = \begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos\alpha & -\sin\alpha \\ 0 & \sin\alpha & \cos\alpha \end{bmatrix}, \qquad R_z(\gamma) = \begin{bmatrix} \cos\gamma & -\sin\gamma & 0 \\ \sin\gamma & \cos\gamma & 0 \\ 0 & 0 & 1 \end{bmatrix}

and $R$ has 3 degrees of freedom. Here the pose is set with a center $C$ and three angles: yaw about the world $Z$ axis, pitch up or down, and roll about the optical axis. The camera's orientation in the world is

R^\top = R_z(\text{yaw}) \, R_x(\text{pitch} - 90^\circ) \, R_z(\text{roll})

The $-90^\circ$ turns the optical axis from straight up to horizontal. $R$ and $t = -R\,C$ are then computed from them.

Conventions. This demo uses the camera frame that is common in computer vision:

x

right,

y

down,

z

forward (into the scene). The pixel origin

(0, 0)

is the top-left corner, with

u

to the right and

v

down. Computer graphics usually has the camera look down

-z

with

y

up, so matrices from the two fields often differ by a flip of the

y

and

z

axes. The world frame is right-handed with

Z

up, and its origin is at a corner of the checkerboard, like a calibration target.

Try this

Increase $f_x$ . The field of view narrows, the image plane moves away from $C$ , and the cube grows in the image.
Set $f_x = 320$ px and check that the horizontal AFOV is $2 \arctan(640 / 640) = 90^\circ$ .
Unlock “fy = fx” and change $f_y$ alone. The image stretches vertically, as with non-square pixels.
Move $u_0$ and watch the frustum become asymmetric. The camera has not moved, but the image content shifts.
Move the camera center $C$ until some vertices end up behind the camera. Points with $Z_c \le 0$ disappear from the image, and the step-by-step panel explains why.
Change only roll and compare $R$ before and after: its third row (the optical axis) stays the same.
Move $C$ to the world origin: $t$ becomes zero. Move it elsewhere and compare $t$ with $C$ . $t = -R\,C$ is the world origin seen from the camera, not the camera position.
Pick a vertex and check the chain $\mathbf{X}_w \to \mathbf{X}_c \to w\,(u, v, 1)^\top \to (u, v)$ by hand.

Pinhole camera: intrinsics & extrinsics

The pinhole model

The projection matrix

Intrinsics KKK

Extrinsics [R∣t][R \mid t][R∣t]

Try this

Intrinsics $K$

Extrinsics $[R \mid t]$