Zhang's Intrinsics from Homographies

Zhang's method is the most widely used technique for estimating camera intrinsics from a planar calibration board. Given homographies from multiple views of the board, it recovers the intrinsics matrix $K$ using constraints from the image of the absolute conic (IAC).

Problem Statement

Given: $M\ge 3$ homographies $\{H_k{\}}_{k=1}^M$ from a planar calibration board to the image, computed from different viewpoints.

Find: Camera intrinsics $K=\left[\begin{array}{ccc}{f}_x& s& c_x\\ 0& f_y& c_y\\ 0& 0& 1\end{array}\right]$.

Assumptions:

• The calibration board is planar
• At least 3 homographies from distinct viewpoints
• The camera intrinsics are constant across all views
• Distortion is negligible (or has been accounted for — see Iterative Intrinsics)

Derivation

Homography and Intrinsics

From the previous chapter, the homography from a $Z=0$ board plane is:

$$H=\lambda K[{\mathbf{r}}_1{\mathbf{r}}_2\mathbf{t}]$$

where $\lambda$ is an arbitrary scale and ${\mathbf{r}}_1,{\mathbf{r}}_2$ are columns of the rotation matrix. Let $H=[{\mathbf{h}}_1{\mathbf{h}}_2{\mathbf{h}}_3]$. Then:

$${\mathbf{h}}_1=\lambda K{\mathbf{r}}_1,{\mathbf{h}}_2=\lambda K{\mathbf{r}}_2$$

Orthogonality Constraints

Since ${\mathbf{r}}_1$ and ${\mathbf{r}}_2$ are columns of a rotation matrix, they satisfy:

$${\mathbf{r}}_1^T{\mathbf{r}}_2=0\text{(orthogonality)}$$ $${\mathbf{r}}_1^T{\mathbf{r}}_1={\mathbf{r}}_2^T{\mathbf{r}}_2\text{(equal norm)}$$

Substituting ${\mathbf{r}}_i=\frac{1}{\lambda }{K}^{-1}{\mathbf{h}}_i$:

$${\mathbf{h}}_1^T{K}^{-T}{K}^{-1}{\mathbf{h}}_2=0$$ $${\mathbf{h}}_1^T{K}^{-T}{K}^{-1}{\mathbf{h}}_1={\mathbf{h}}_2^T{K}^{-T}{K}^{-1}{\mathbf{h}}_2$$

The Image of the Absolute Conic (IAC)

Define the symmetric matrix:

$$B=K^{-T}{K}^{-1}=\left[\begin{array}{ccc}{B}_{11}& B_{12}& B_{13}\\ B_{12}& B_{22}& B_{23}\\ B_{13}& B_{23}& B_{33}\end{array}\right]$$

Since $B$ is symmetric, it has 6 independent entries. The constraints become:

$${\mathbf{h}}_1^{T}B{\mathbf{h}}_2=0$$ $${\mathbf{h}}_1^{T}B{\mathbf{h}}_1-{\mathbf{h}}_2^{T}B{\mathbf{h}}_2=0$$

Each homography gives 2 linear equations in the 6 unknowns of $B$.

The ${\mathbf{v}}_{ij}$ Vectors

To write the constraints as a linear system, define the 6-vector:

$${\mathbf{v}}_{ij}=\left[\begin{array}{c}{h}_{1i}{h}_{1j}\\ h_{1i}{h}_{2j}+h_{2i}{h}_{1j}\\ h_{2i}{h}_{2j}\\ h_{3i}{h}_{1j}+h_{1i}{h}_{3j}\\ h_{3i}{h}_{2j}+h_{2i}{h}_{3j}\\ h_{3i}{h}_{3j}\end{array}\right]$$

where $h_{ki}$ is the $(k,i)$-th element of $H$ (1-indexed, column $i$, row $k$). Then:

$${\mathbf{h}}_i^{T}B{\mathbf{h}}_j={\mathbf{v}}_{ij}^T\mathbf{b}$$

where $\mathbf{b}=[B_{11},B_{12},B_{22},B_{13},B_{23},B_{33}{]}^T$.

The Linear System

For each homography $H_k$, the two constraints become:

$$\left[\begin{array}{c}{\mathbf{v}}_{12}^T\\ ({\mathbf{v}}_{11}-{\mathbf{v}}_{22}{)}^T\end{array}\right]\mathbf{b}=0$$

Stacking $M$ homographies gives a $2M\times 6$ system:

$$V\mathbf{b}=0$$

With $M\ge 3$ (giving $\ge 6$ equations), this is solved via SVD: $\mathbf{b}$ is the right singular vector of $V$ corresponding to the smallest singular value.

Extracting Intrinsics from $B$

Given $\mathbf{b}=[B_{11},B_{12},B_{22},B_{13},B_{23},B_{33}]$, the intrinsics are recovered by a Cholesky-like factorization of $B=K^{-T}{K}^{-1}$.

The closed-form extraction:

$$v_0=\frac{B_{12}{B}_{13}-B_{11}{B}_{23}}{B_{11}{B}_{22}-B_{12}^2}$$

$$\lambda =B_{33}-\frac{B_{13}^2+v_0(B_{12}{B}_{13}-B_{11}{B}_{23})}{B_{11}}$$

$$f_x=\sqrt{\frac{\lambda }{B_{11}}}$$

$$f_y=\sqrt{\frac{\lambda B_{11}}{B_{11}{B}_{22}-B_{12}^2}}$$

$$s=-\frac{B_{12}{f}_x^2{f}_y}{\lambda }$$

$$c_x=\frac{s\cdot v_0}{f_y}-\frac{B_{13}{f}_x^2}{\lambda }$$

$$c_y=v_0$$

Validity Checks

$B=\mu K^{-T}{K}^{-1}$ is definite, so the extraction fails when:

• $B_{11}{B}_{22}-B_{12}^2$ is not positive, or $|B_{11}|\approx 0$
• $\lambda$ and $B_{11}$ have different signs (negative focal length squared)

Either way no camera fits the homographies, and the solve returns DegenerateViews: the views do not constrain the intrinsics, typically because the board planes are parallel in every view (all fronto-parallel, or all tilted the same way).

Parallel Boards

A board's vanishing line is ${\mathbf{h}}_1\times {\mathbf{h}}_2\propto K^{-T}{\mathbf{r}}_3$, so parallel boards share one vanishing line and give every view the same two constraints: $V$ has rank 2. With noisy corners the solve can then return an arbitrary $K$ rather than fail. The dataset-level initializer (estimate_intrinsics_iterative) therefore checks the views first: it takes $K_0^T({\mathbf{h}}_1\times {\mathbf{h}}_2)$ as each board's normal, with $K_0$ a square-pixel camera centred on the observed corners whose focal length is their extent, and returns DegenerateViews when no normal lies more than 1° from the mean plane. Parallel boards read zero through any $K_0$, up to the corner noise (about 0.15° for a board 40 % of the image wide and 0.2 px of noise); a focal length of the image extent reads the perspective the views show, which is what the constraints feed on.

Minimum Number of Views

• $M=3$ homographies: 6 equations for the 6 unknowns of $B$ — the minimum for a unique solution. The solve also yields a skew term $\gamma$; it is needed to recover $c_x$ and then discarded, because the camera model has no skew
• $M=2$: Only 4 equations, not enough for the general solve
• $M>3$: Overdetermined system; SVD gives the least-squares solution

Practical Considerations

View diversity: The views should include significant rotation around both axes. Pure translations leave the board planes parallel, and the intrinsics undetermined. Two orientations only, such as tilts about one axis both ways, give four constraints for the five unknowns, and leave one focal length nearly free; three distinct tilts about one axis determine the camera.

Distortion: Zhang's method assumes distortion-free observations. When applied to distorted pixels, the estimated $K$ is biased. The Iterative Intrinsics chapter addresses this with an alternating refinement scheme.

OpenCV equivalence: Zhang's method is the internal initialization step of cv::calibrateCamera. OpenCV does not expose it as a separate API.

API

#![allow(unused)]
fn main() {
let K = estimate_intrinsics_from_homographies(&homographies)?;
}

Returns FxFyCxCy with the estimated intrinsics. Typically followed by distortion estimation and non-linear refinement.

Reference

Zhang, Z. (2000). "A Flexible New Technique for Camera Calibration." IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1330-1334.