Solver Backends

Every non-linear solve in calibration-rs runs one Levenberg–Marquardt (LM) loop. A backend supplies what the loop needs at the current parameters: the residuals and their Jacobian with the robust loss folded in, the robust objective, and the retraction that applies a step to each parameter manifold. Two backends are available:

• tiny-solver (the default): forward-mode autodiff with dynamic-size dual numbers; a robust loss enters through the Triggs correction.
• factrs: forward-mode autodiff with static-size dual numbers; a robust loss enters through iterative reweighting.

Both evaluate the same residual kernels, retract SE(3) and S² the same way, and minimize the same objective, so they reach the same minimizer. They differ in how fast they build the Jacobian and, on robust problems, in the path they take to the minimum.

Selecting a Backend

The backend is the backend field of the solver settings of every workflow.

Rust:

#![allow(unused)]
fn main() {
use vision_calibration::optim::SolverBackend;
use vision_calibration::planar_intrinsics::PlanarIntrinsicsConfig;

let mut config = PlanarIntrinsicsConfig::default();
config.solver.backend = SolverBackend::Factrs;
}

JSON config:

{ "solver": { "max_iters": 50, "robust_loss": "None", "backend": "factrs" } }

Python:

config = vc.PlanarIntrinsicsConfig(solver=vc.SolverConfig(backend="factrs"))

In the desktop app, the backend is a field of the solver group of each config form. Workflows with several solve stages (for example the joint rig, hand-eye and laser workflow) have one solver group per stage, each with its own backend.

In code, set it on the config's solver group:

#![allow(unused)]
fn main() {
let mut config = PlanarIntrinsicsConfig::default();
config.solver.backend = SolverBackend::Factrs;
}

The Levenberg–Marquardt Loop

The objective is the robust cost

$$F(\mathbf{x})=\frac{1}{2}\sum _i{\rho }_i\left(\Vert {\mathbf{r}}_i(\mathbf{x}){\Vert }^2\right),$$

where ${\rho }_i$ is the residual block's loss ($\rho (s)=s$ without one; see Robust Loss Functions). Each iteration:

1. Linearizes at $\mathbf{x}$: the backend returns a residual $\mathbf{r}$ and Jacobian $J$ with the loss folded in, so that the model $\Vert \mathbf{r}+J\mathbf{\delta }{\Vert }^2$ has the same first-order change as $2F$.
2. Scales the Jacobian's columns by $1/(1+\Vert {\tilde{J}}_c\Vert )$, computed once at the first iteration, so that parameters of different magnitude (focal lengths in pixels, distortion coefficients, rotations) are damped comparably.
3. Solves the damped normal equations $(J^{T}J+\lambda D)\mathbf{\delta }=-J^T\mathbf{r}$ with sparse Cholesky, where $D$ is the diagonal of $J^{T}J$ clamped to $[10^{-6},10^{32}]$. The sparsity pattern of $J^{T}J$ and the symbolic factorization are computed once per solve; each iteration fills the values of the lower triangle only.
4. Applies the step through each block's retraction (vector addition, $\mathbf{x}\cdot \exp (\mathbf{\delta })$ on SE(3), the exponential map on S²), then clamps bounded parameters to their bounds.
5. Accepts or rejects by the gain ratio of the actual to the predicted decrease of $F$. An accepted step scales $\lambda$ by $\max (1/3,1-(2\varrho -1{)}^3)$; a rejected one doubles $\lambda$ and retries, up to 32 times.

The loop stops when $F$ falls below min_error, when its absolute or relative decrease falls below min_abs_decrease / min_rel_decrease, when a step can no longer change the parameters ($\Vert \mathbf{\delta }\Vert \le 10^{-8}(\Vert \mathbf{x}\Vert +10^{-8})$), when no step is accepted after 32 retries, or after max_iters iterations. Parameters held fixed never move, and the same input gives bit-identical results run to run.

How the Backends Differ

Without a robust loss the two linearizations are the same, and so is every step up to rounding. With a robust loss they take different steps towards the same minimum, so iteration counts can differ.

factrs supports what the calibration problems use. It rejects an IR it cannot map, with an InvalidInput error: a partially fixed SE(3) or S² block, bounds on a non-vector block, or one camera's parameter blocks combined differently in two factors.

Performance and Quality

Measured on a synthetic benchmark of the seven workflows, with planar intrinsics and rig extrinsics each run with a straight and a tilted (Scheimpflug) sensor: 150 scenes across three sizes (8 to 36 views), two pixel-noise levels and 5 % outliers under Huber or Cauchy losses, timed as the median of three runs on an Apple M4 Pro (8 performance + 4 efficiency cores). Times are the median non-linear solve per scene, all stages included.

Quality is the same. The backends end at the same final cost (the largest relative difference is 2e-4, typically below 1e-7), take the same number of iterations in 137 of 144 scenes, and recover the same parameters.

Speed depends on the cores available. tiny-solver evaluates residual blocks in parallel; factrs evaluates them on one thread. On a single thread factrs is 2.4–3.3× faster (measured on the planar intrinsics, laserline device and rig hand-eye scenes); its dual numbers have a fixed size and live on the stack, while tiny-solver's are allocated on the heap. With all cores, tiny-solver's 3.7–5.7× parallel speed-up wins.

Which to use. tiny-solver, the default, is faster whenever the solve has the machine's cores to itself. factrs uses a core more efficiently, so it is the better choice when cores are scarce: a single-threaded deployment, or an application running many calibrations at once.

Solver Options

Each solve stage's SolverConfig sets the backend, the iteration budget max_iters, the verbosity and the robust_loss. The loop stops when the cost decreases by less than 1e-5 in absolute or relative terms, or falls below 1e-10.

The damped normal equations are always solved by sparse Cholesky. When the factorization fails (the damped $J^{T}J$ is not numerically positive definite), the loop treats it as a rejected step: it raises the damping and retries.

Solve Report

#![allow(unused)]
fn main() {
pub struct SolveReport {
    pub final_cost: f64,
    pub num_iters: usize,
}
}

final_cost is the robust objective $F$ at the solution, the same definition for both backends, so it is comparable between them. num_iters counts the LM iterations.

Typical Convergence

For a well-initialized planar intrinsics problem:

• Final cost: $\sim 10^{-2}$–$10^0$ (sub-pixel residuals)
• Iterations: 5–50 (depends on problem size and initial quality)
• Termination: usually the relative decrease falling below min_rel_decrease

Error Handling

A solve fails with an error for:

• a missing or wrongly sized initial value for a parameter block;
• an invalid IR (for example a non-positive robust-loss scale);
• an IR the selected backend cannot map (see above);
• a non-finite initial cost.