Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

calib-targets-rs is a workspace of Rust crates for detecting and modeling planar calibration targets from corner clouds (for example, ChESS corners). The focus is geometry-first: target modeling, grid fitting, and rectification live here, while image I/O and corner detection are intentionally out of scope.

ChArUco detection overlay ChArUco detection overlay on a small board.

What it is:

  • A small, composable set of crates for chessboard, ChArUco, and marker-style targets.
  • A set of geometric primitives (homographies, rectified views, grid coords).
  • Practical examples and tests based on the chess-corners crate.

What it is not:

  • A replacement for your corner detector or image pipeline.
  • A full calibration stack (no camera calibration or PnP here).

Recommended reading order:

  1. Project Overview and Conventions
  2. Pipeline Overview
  3. Crate chapters, starting with calib-targets-core and calib-targets-chessboard

API docs.

Quickstart

Install the facade crate (the image feature is enabled by default):

cargo add calib-targets image

Minimal chessboard detection:

use calib_targets::detect;
use calib_targets::chessboard::ChessboardParams;
use image::ImageReader;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let img = ImageReader::open("board.png")?.decode()?.to_luma8();
    let chess_cfg = detect::default_chess_config();
    let params = ChessboardParams::default();

    let result = detect::detect_chessboard(&img, &chess_cfg, params);
    println!("detected: {}", result.is_some());
    Ok(())
}

Python bindings

Python bindings are built with maturin:

pip install maturin
maturin develop
python crates/calib-targets-py/examples/detect_chessboard.py path/to/image.png

The calib_targets module exposes detect_chessboard, detect_charuco, and detect_marker_board. Config inputs accept either typed classes (for IDE autocomplete) or dict overrides. detect_charuco requires params and the board lives in params.board. For marker boards, target_position is populated only when params.layout.cell_size (or params["layout"]["cell_size"]) is set and alignment succeeds.

MSRV: Rust 1.70 (stable).

Project Overview

calib-targets-rs is a single Cargo workspace with multiple publishable crates under crates/. The design is layered: calib-targets-core provides geometry and shared types, higher-level crates build on top, and the facade crate (calib-targets) is intended to be the main entry point.

Mesh-rectified grid

Workspace layout

  • calib-targets-core: shared geometry types and utilities.
  • calib-targets-chessboard: chessboard detection from corner clouds.
  • calib-targets-aruco: embedded dictionaries and decoding on rectified grids.
  • calib-targets-charuco: grid-first ChArUco detector and alignment.
  • calib-targets-marker: checkerboard marker detector (chessboard + circles).
  • calib-targets: facade crate, currently hosting examples and future high-level APIs.

Strengths

  • Clear crate boundaries with a small, geometry-first core.
  • Chessboard detection pipeline is implemented end-to-end with debug outputs.
  • Mesh-warp rectification supports lens distortion without assuming a single global homography.
  • Examples and regression tests exist for all workflows.

Gaps and early-stage areas

  • Public APIs are not yet stable.
  • ArUco decoding assumes rectified grids and does not perform quad detection.
  • Performance/benchmarks are not yet a focus.

Conventions

These conventions are used throughout the workspace. They are not optional and should not change silently.

Coordinate systems

  • Image pixels: origin at top-left, x increases right, y increases down.
  • Grid coordinates: i increases right, j increases down.
  • Grid indices in detections are corner indices (intersections), not square indices, unless explicitly stated otherwise.

Homography and quad ordering

  • Quad corner order is always TL, TR, BR, BL (clockwise).
  • The ordering must match in both source and destination spaces.
  • Never use self-crossing orders like TL, TR, BL, BR.

Sampling and pixel centers

  • Warping and sampling should be consistent about pixel centers.
  • When in doubt, treat sample locations as (x + 0.5, y + 0.5) in pixel space.

Orientation angles

  • ChESS-style corner orientations are in radians and defined modulo pi (not 2*pi).
  • Orientation clustering finds two dominant directions and assigns each corner to cluster 0 or 1, or marks it as an outlier.

Marker bit conventions

  • Marker codes are packed in row-major order.
  • Black pixels represent bit value 1.
  • Border width is defined in whole cells (border_bits).

If you introduce new algorithms or data structures, document any additional conventions in the relevant crate chapter.

Pipeline Overview

At a high level, the workflow looks like this:

  1. Input corners: supply a list of calib_targets_core::Corner values, typically from a ChESS detector.
  2. Estimate grid axes: cluster corner orientations to get two dominant grid directions.
  3. Build a grid graph: connect corners that plausibly lie on the same grid lines.
  4. Assign integer coordinates: BFS the graph to produce (i, j) grid indices.
  5. Select the best board: choose the best connected component that matches expected size.
  6. Rectify (optional): compute a global homography or mesh warp to build a rectified view.
  7. Decode markers (optional): decode per-cell directly, or scan a rectified grid if you need a full rectified image.
  8. Align board (optional): map markers to a known layout and assign corner IDs.

Output types are standardized in calib-targets-core as TargetDetection with LabeledCorner values. Higher-level crates enrich that output with additional metadata (inliers, marker detections, rectified views).

Crates

The workspace is organized as a stack of crates with minimal, composable boundaries.

Dependency direction

  • calib-targets-core is the base and should not depend on higher-level crates.
  • calib-targets-chessboard depends on core for geometry and types.
  • calib-targets-aruco depends on core for rectified image access.
  • calib-targets-charuco depends on chessboard and aruco.
  • calib-targets-marker depends on chessboard and core.
  • calib-targets is the facade that re-exports types and offers end-to-end helpers.

Python bindings

Python bindings are provided by the calib-targets-py crate (module name calib_targets). It depends on the facade crate and is built with maturin; see crates/calib-targets-py/README.md in the repository root.

Where to start

If you are new to the codebase, start with:

  1. calib-targets-core
  2. calib-targets-chessboard

Then branch into the target-specific crates depending on your use case.

calib-targets-core

calib-targets-core provides shared geometric types and utilities. It is intentionally small and purely geometric; it does not depend on any particular corner detector or image pipeline.

Rectified grid view Global rectification output from the chessboard pipeline.

Core data types

  • Corner: raw corner observations from your detector.
    • position: image-space pixel coordinates.
    • orientation: dominant grid orientation in radians, defined modulo pi.
    • orientation_cluster: optional cluster index (0 or 1) if clustering is used.
    • strength: detector response.
  • GridCoords: integer grid indices (i, j) in board space.
  • LabeledCorner: a detected corner with optional grid coordinates and logical ID.
  • TargetDetection: a collection of labeled corners for one board instance.
  • TargetKind: enum for Chessboard, Charuco, or CheckerboardMarker.

These types are shared across all detectors so downstream code can be target-agnostic.

Orientation clustering

ChESS corner orientations are only defined modulo pi. The clustering utilities help recover two dominant grid directions:

  • cluster_orientations: histogram-based peak finding followed by 2-means refinement.
  • OrientationClusteringParams: histogram size, separation thresholds, outlier rejection.
  • compute_orientation_histogram: debug visualization helper.
  • estimate_grid_axes_from_orientations: a lightweight fallback when clustering fails.

Chessboard detection uses these helpers to label corners by axis and to reject outliers.

Homography and rectification

Homography is a small wrapper around a 3x3 matrix with helpers for DLT estimation and point mapping:

  • estimate_homography_rect_to_img: DLT with Hartley normalization for N >= 4 point pairs.
  • homography_from_4pt: direct 4-point estimation.
  • warp_perspective_gray: warp a grayscale image using a homography.

For mapping rectified pixels back to the original image, core defines:

  • RectifiedView: a rectified grayscale image and its mapping info.
  • RectToImgMapper: either a single global homography or a per-cell mesh map.

Higher-level crates (notably chessboard) wrap these utilities for global or mesh rectification.

Image utilities

GrayImage and GrayImageView are lightweight, row-major grayscale buffers with bilinear sampling helpers:

  • sample_bilinear: float sampling with edge clamp to 0.
  • sample_bilinear_u8: u8 sampling with clamping to 0..255.

These utilities are used by rectification and marker decoding.

Conventions recap

  • Coordinate system: origin at top-left, x right, y down.
  • Grid coordinates: i right, j down, and grid indices refer to corners.
  • Quad order: TL, TR, BR, BL in both source and destination spaces.

If you build on core types, stick to these conventions to avoid subtle alignment bugs.

calib-targets-chessboard

calib-targets-chessboard detects a plain chessboard from a cloud of ChESS corners. It is graph-based and perspective-aware, and it returns integer grid coordinates for each detected corner.

Chessboard detection overlay Detected chessboard corners overlaid on the source image.

Detection pipeline

The detector follows these steps (see ChessboardDetector):

  1. Filter corners by minimum strength.
  2. Estimate two dominant grid axes from corner orientations.
  3. Estimate a base spacing from nearest-neighbor distances.
  4. For each corner, find up to 4 neighbors (right/left/up/down) based on distance and orientation consistency.
  5. Build a 4-connected undirected grid graph.
  6. BFS each connected component and assign integer (i, j) coordinates.
  7. Compute width, height, and completeness per component.
  8. Keep the best component that matches expected size and completeness thresholds.

Currently the detector returns at most one board instance (the best-scoring component).

Key types

  • ChessboardDetector: main entry point.
  • ChessboardParams: detection thresholds and expected board size.
  • GridGraphParams: neighbor search and geometric constraints.
  • ChessboardDetectionResult:
    • detection: TargetDetection with labeled corners.
    • inliers: indices into the corner list used for rectification.
    • orientations: estimated grid axes (optional).
    • debug: optional histogram and graph data for diagnostics.

Parameters

ChessboardParams controls high-level validity checks:

  • min_corner_strength: filter weak corners early.
  • min_corners: minimum number of corners to accept a component.
  • expected_rows, expected_cols: inner corner counts in each direction.
  • completeness_threshold: detected / expected corner ratio.
  • use_orientation_clustering: toggle orientation clustering (enabled by default).

GridGraphParams controls how neighbors are chosen:

  • min_spacing_pix, max_spacing_pix: expected corner spacing range in pixels.
  • k_neighbors: how many nearest neighbors to consider.
  • orientation_tolerance_deg: angular tolerance for neighbor relations.

Grid graph details

Neighbor selection uses orientation information in two modes:

  • With clustering: corners are labeled by one of two axis clusters. Candidate edges must align with one of the two grid directions derived from those clusters.
  • Without clustering: orientations are checked for near-orthogonality, and the edge direction must be close to 45 degrees from each corner orientation.

Edges are classified into Right, Left, Up, Down based on image-space directions, and only the best candidate per direction is kept. This yields a clean 4-connected grid graph for BFS.

Rectification helpers

The crate provides two rectification options:

  • rectify_from_chessboard_result: fits a single global homography and produces a RectifiedBoardView.
  • rectify_mesh_from_grid: fits one homography per cell and produces a RectifiedMeshView (more robust to lens distortion).

Both require labeled corners and a chosen px_per_square scale.

Example

#![allow(unused)]
fn main() {
use calib_targets_chessboard::{ChessboardDetector, ChessboardParams};
use calib_targets_core::Corner;

fn detect(corners: &[Corner]) {
    let params = ChessboardParams::default();
    let detector = ChessboardDetector::new(params);

    if let Some(result) = detector.detect_from_corners(corners) {
        println!("detected {} corners", result.detection.corners.len());
    }
}
}

For a full runnable example, see crates/calib-targets-chessboard/examples/chessboard.rs.

calib-targets-aruco

calib-targets-aruco provides embedded ArUco/AprilTag-style dictionaries and decoding on rectified grids. It does not detect quads or perform image rectification by itself.

Mesh-rectified grid Rectified grid used for ArUco/AprilTag decoding.

Current API surface

  • Dictionary: built-in dictionary metadata and packed codes.
  • Matcher: brute-force matching against a dictionary with rotation handling.
  • ScanDecodeConfig: how to scan a rectified grid (border size, inset, polarity).
  • scan_decode_markers: read and decode markers from rectified cells.
  • scan_decode_markers_in_cells: decode markers from per-cell image quads (no full warp).
  • decode_marker_in_cell: decode a single marker inside one square cell.

The crate expects a rectified view where each chessboard square is approximately px_per_square pixels and where cell indices align with the board grid.

Decoding paths

There are two supported scanning modes:

  • Rectified grid scan (scan_decode_markers): build a rectified image first and scan a regular grid.
  • Per-cell scan (scan_decode_markers_in_cells): pass a list of per-cell quads and decode each cell directly.

Per-cell scanning avoids building the full rectified image and is easy to parallelize across cells.

Scan configuration

ScanDecodeConfig controls how bit sampling and thresholding behave:

  • border_bits: number of black border cells (OpenCV typically uses 1).
  • marker_size_rel: marker size relative to the square size (ChArUco uses < 1.0).
  • inset_frac: extra inset inside the marker to avoid edge blur.
  • min_border_score: minimum fraction of border bits that must be black.
  • dedup_by_id: keep only the best detection per marker id.

If decoding is too sparse on real images, reduce inset_frac slightly and re-run.

Conventions

  • Marker bits are packed row-major with black = 1.
  • Match::rotation is in 0..=3 such that observed == rotate(dict_code, rotation).
  • border_bits matches the OpenCV definition (typically 1).

Status

Decoding is implemented and stable for rectified grids, but quad detection and image-space marker detection are deliberately out of scope. The roadmap chapter details planned improvements and API refinements.

For deeper tuning and sampling details, see ArUco Decoding Details.

ArUco Decoding Details

This chapter expands on the marker decoding path in calib-targets-aruco. The decoder is grid-first: it samples expected square cells and reads bits in rectified space (or per-cell quads).

When to use per-cell decoding

Use per-cell decoding (scan_decode_markers_in_cells) when you already have a grid of square corners and want to avoid warping the full image. It works well with ChArUco detection because you can decode only the valid cells and parallelize across them.

Sampling model

  • Bits are sampled on a regular grid inside the marker area.
  • The marker area is defined by marker_size_rel, with an extra inset from inset_frac.
  • A per-marker threshold (Otsu) is computed from sampled intensities.

Tuning checklist

  • If markers are missing, try reducing inset_frac slightly.
  • If false positives appear, raise min_border_score or enable dedup_by_id.
  • Make sure marker_size_rel matches the physical board spec.

calib-targets-charuco

calib-targets-charuco combines chessboard detection with ArUco decoding to detect ChArUco boards. ChArUco dictionaries and board layouts are fully compatible with OpenCV’s aruco/charuco implementation. The flow is grid-first:

ChArUco detection overlay ChArUco detection overlay with assigned corner IDs.

  1. Detect a chessboard grid from ChESS corners.
  2. Build per-cell quads from the detected grid.
  3. Decode markers per cell (no full-image warp).
  4. Align marker detections to a board specification and assign corner IDs.

Board specification

  • CharucoBoardSpec describes the board geometry:
    • rows, cols are square counts (not inner corners).
    • cell_size is the physical square size.
    • marker_size_rel is the marker size relative to a square.
    • dictionary selects the marker dictionary.
    • marker_layout defines the placement scheme.
  • CharucoBoard validates and precomputes marker placement.

Detector

  • CharucoDetectorParams::for_board provides a reasonable default configuration.
  • CharucoDetector::detect returns a CharucoDetectionResult with:
    • detection: labeled corners with ChArUco IDs, filtered to marker-supported corners.
    • markers: decoded marker detections in rectified grid coordinates (with optional corners_img).
    • alignment: grid alignment from detected grid coordinates into board coordinates.

Per-cell decoding

The detector decodes markers per grid cell. This avoids building a full rectified image and keeps the work proportional to the number of valid squares. If you need a full rectified image for visualization, use the rectification helpers in calib-targets-chessboard on a detected grid.

Alignment and refinement

Alignment maps decoded marker IDs to board positions using a small set of grid transforms and a translation vote. Once an alignment is found, the detector re-decodes markers at their expected cell locations and re-solves the alignment to filter out inconsistencies.

This two-stage approach helps reject spurious markers while keeping the final corner IDs consistent.

Tuning notes

  • scan.inset_frac trades off robustness vs. sensitivity. The defaults in for_board use a slightly smaller inset (0.06) to improve real-image decoding.
  • min_marker_inliers controls how many aligned markers are required to accept a detection.

Status

The current implementation focuses on the OpenCV-style layout and is intentionally conservative about alignment. Extensions for more layouts and improved robustness are planned (see the roadmap).

For alignment details, see ChArUco Alignment and Refinement.

ChArUco Alignment

calib-targets-charuco aligns decoded marker IDs to a board layout and assigns ChArUco corner IDs. Alignment is discrete and fast: it tries a small set of grid transforms and selects the translation with the strongest inlier support.

Alignment pass

  • Each decoded marker votes for a board translation under each candidate transform.
  • The best translation wins (ties broken by inlier count).
  • Inliers are the markers whose (sx, sy) map exactly to the expected board cell for their ID.

Inlier filtering

After alignment is chosen, the detector keeps only inlier markers and assigns ChArUco corner IDs based on the aligned grid coordinates. The final alignment in the result is a GridAlignment that maps detected grid coordinates into board coordinates.

calib-targets-marker

calib-targets-marker targets a checkerboard marker board: a chessboard grid with three circular markers near the center. The detector is grid-first and works with partial boards.

Marker-board detection overlay Detected circle markers and aligned grid overlay.

Detection pipeline

  1. Chessboard detection: run calib-targets-chessboard to obtain grid-labeled corners (partial boards are allowed).
  2. Per-cell circle scoring: for every valid square cell, warp the cell to a canonical patch and score a circle by comparing a disk sample to an annular ring.
  3. Candidate filtering: keep the strongest circle candidates per polarity.
  4. Circle matching: match candidates to the expected layout (cell coordinates + polarity).
  5. Grid alignment estimation: derive a dihedral transform + translation from detected grid coordinates to board coordinates when enough circles agree.

Key types

  • MarkerBoardDetector: main entry point.
  • MarkerBoardLayout: rows/cols plus the three expected circles (cell coordinate + polarity).
  • MarkerBoardParams: layout + chessboard/grid graph params + circle score + match settings.
  • MarkerBoardDetectionResult:
    • detection: TargetDetection labeled as CheckerboardMarker.
    • circle_candidates: scored circles per cell.
    • circle_matches: matched circles (with offsets).
    • alignment: optional GridAlignment from detected grid coords to board coords.
    • alignment_inliers: number of circle matches used for the alignment.

Parameters

MarkerBoardLayout defines the board and marker placement:

  • rows, cols: inner corner counts.
  • cell_size: optional square size in your world units (when set, target_position is populated).
  • circles: three MarkerCircleSpec entries with cell (top-left corner indices) and polarity.

MarkerBoardParams configures detection:

  • chessboard: ChessboardParams (defaults to completeness_threshold = 0.05 to allow partial boards).
  • grid_graph: GridGraphParams for neighbor search constraints.
  • circle_score: per-cell circle scoring parameters.
  • match_params: candidate filtering and matching thresholds.
  • roi_cells: optional cell ROI [i0, j0, i1, j1].

CircleScoreParams controls scoring:

  • patch_size: canonical square size in pixels.
  • diameter_frac: circle diameter relative to the square.
  • ring_thickness_frac: ring thickness relative to circle radius.
  • ring_radius_mul: ring radius relative to circle radius.
  • min_contrast: minimum accepted disk-vs-ring contrast.
  • samples: samples per ring for averaging.
  • center_search_px: small pixel search around the cell center.

CircleMatchParams controls matching:

  • max_candidates_per_polarity: top-N candidates to keep per polarity.
  • max_distance_cells: optional maximum distance for a match.
  • min_offset_inliers: minimum agreeing circles to return an alignment.

Notes

  • Cell coordinates (i, j) refer to square cells, expressed by the top-left corner indices. The cell center is at (i + 0.5, j + 0.5).
  • alignment maps detected grid coordinates into board coordinates using a dihedral transform and translation.

calib-targets (facade)

The calib-targets crate is the unified entry point for the workspace. It re-exports the lower-level crates and provides optional end-to-end helpers in calib_targets::detect (feature image, enabled by default).

Mesh-rectified grid Facade examples cover detection and rectification workflows.

Current contents

  • Re-exports: core, chessboard, aruco, charuco, marker.
  • detect module: helpers that run ChESS corner detection and then the target detector.
  • Examples under crates/calib-targets/examples/ that take an image path.

Features

  • image (default): enables calib_targets::detect.
  • tracing: enables tracing output across the subcrates.

See the roadmap for future expansion of the facade API.

Examples

Examples live under crates/*/examples/ and are built per crate. Many examples accept a JSON config file (defaults point to testdata/ or tmpdata/), while the facade examples under calib-targets take an image path directly.

To run an example from the workspace root:

cargo run -p calib-targets-chessboard --example chessboard -- testdata/chessboard_config.json

Python examples live under crates/calib-targets-py/examples/ and use the calib_targets module. After maturin develop, run them with an image path, for example:

python crates/calib-targets-py/examples/detect_charuco.py testdata/small2.png

See the sub-chapters for what each example produces and how to interpret the outputs.

Chessboard Detection Example

File: crates/calib-targets-chessboard/examples/chessboard.rs

This example runs the full chessboard pipeline:

Chessboard detection overlay Example output overlay for chessboard detection.

  1. Detects ChESS corners using the chess-corners crate.
  2. Adapts them to calib_targets_core::Corner.
  3. Runs ChessboardDetector.
  4. Optionally outputs debug data (orientation histogram, grid graph).

The default config is testdata/chessboard_config.json (input: testdata/mid.png, output: tmpdata/chessboard_detection_mid.json).

Run it with:

cargo run -p calib-targets-chessboard --example chessboard -- testdata/chessboard_config.json

The output JSON contains detected corners, grid coordinates, and optional debug diagnostics (if debug_outputs are enabled in the config).

Global Rectification Example

File: crates/calib-targets-chessboard/examples/rectify_global.rs

This example detects a chessboard and computes a single global homography to produce a rectified board view. The output includes:

Global rectification output Global rectification output from the small test image.

  • A rectified grayscale image.
  • A JSON report with homography matrices and grid bounds.

The code defaults to tmpdata/rectify_config.json, but a ready-made config exists in testdata/rectify_config.json (input: testdata/small.png, rectified output: tmpdata/rectified_small.png, report: tmpdata/charuco_report_small.json).

Run it with:

cargo run -p calib-targets-chessboard --example rectify_global -- testdata/rectify_config.json

If rectification succeeds, the rectified image is written to tmpdata/rectified.png unless overridden in the config.

Mesh Rectification Example

File: crates/calib-targets-aruco/examples/rectify_mesh.rs

This example detects a chessboard, performs per-cell mesh rectification, and scans the rectified grid for ArUco markers. It writes:

Mesh rectification output Per-cell mesh rectification output from the small test image.

  • A mesh-rectified grayscale image.
  • A JSON report with rectification info and marker detections.

The code defaults to testdata/rectify_mesh_config_small0.json, and that config is a good starting point (input: testdata/small0.png, mesh output: tmpdata/mesh_rectified_small0.png, report: tmpdata/rectify_mesh_report_small0.json).

Run it with:

cargo run -p calib-targets-aruco --example rectify_mesh -- testdata/rectify_mesh_config_small0.json

This is a good reference for the full grid -> rectification -> marker scan pipeline.

Data and Tools

This repository includes data and scripts that support testing and debugging.

Data folders

  • testdata/: sample images and JSON configs used by examples and tests.

Tools

The tools/ folder contains helper scripts for visualization and synthetic data generation, for example:

  • Overlay plotting for chessboard, marker, and ChArUco outputs.
  • Synthetic marker target generation.
  • Dictionary utilities.

These tools are optional but useful when debugging or extending detectors.

Roadmap

To be added later