Skip to content

5. Performance: how to use it most efficiently

Numbers below: Intel i5-12400 (6 cores / 12 threads), published wheels (OpenCV 4.10, AVX2), FRACTAL_5L_6, a marker in a cluttered scene, median per frame. Measure your own machine with the commands at the end.

Decision table

situation do this why
live video, one camera FractalTracker(detector), default threads searches near the last detection: ~2× at 4K; all cores inside each frame
live 4K video, marker not tiny FractalTracker(detector, auto_scale=True) also shrinks the region from the last marker size: ×3.9 (all cores) to ×5.7 (1 core) at 4K, same corners
marker near the edge of range flickers FractalTracker(detector, use_prior=True) keeps a level read a moment ago; ~0.06 ms, nothing without a previous result
the height is known (rangefinder) detect(..., expected_side_px=(lo, hi)) reads the marker's size with a looser tolerance and returns lookalikes; +0.2–0.3 ms per frame (one core)
marker in view on the approach but never read FractalTracker(detector, capture_frames=3) picks it up once it holds its place; +4–22 % per frame while there is no track (+0.06 ms at 640×480, +0.77 ms at 1080p, one core), nothing once tracking
every frame has a marker or none matters, and microseconds count params.close_pass = False saves the second threshold pass on empty frames (0.25–0.5 ms on one core at 720p–1080p); loses markers seen from very close on dark frames
4K (or larger) frames, marker not tiny params.detection_scale = 0.5 threshold/contours on a quarter of the pixels; corners still refined at full resolution
several cameras / streams at once nf.set_num_threads(1), one detector (or tracker) per stream streams run in parallel instead of competing for cores
a folder of images / recorded video detector.detect_batch(frames) one frame per core, best throughput
marker position known roughly detect(image, roi=(x, y, w, h)) only that region is processed
need pose with_inner_points=True costs ~1.5 ms extra at 1080p, much better pose
only need to know where the marker is plain detect() skip the inner-point stage
colour camera pass gray if you have it; BGR is fine too conversion is only ~0.07 ms at 1080p

Measured

1080p 4K
detect(), all cores 0.9–1.1 ms 3.7–4.2 ms
detect(), 1 core 3.2 ms 12.5 ms
detect(with_inner_points=True), all cores 2.3–2.5 ms 7.4 ms
FractalTracker, all cores (moving marker) 0.57 ms 2.2 ms
detection_scale=0.5, all cores 0.37 ms 0.94 ms
detect_batch, 48 frames ~1360 frames/s —

Version 0.5 needed 12–14 ms at 1080p and ~50 ms at 4K for the same frames.

detection_scale

detection_scale runs the expensive stages (adaptive threshold, contour tracing, decoding) on a downscaled copy, then refines the corners on the full-resolution image, so accuracy stays sub-pixel as long as the markers stay large enough after scaling. Any value in (0, 1) works; values ≥ 1 (and ≤ 0) mean "no scaling".

4K frame, FRACTAL_5L_6, wheels' OpenCV 4.10:

scale all cores 1 core levels found
1.0 4.3 ms 12.2 ms 0, 1, 2, 3
0.75 2.6 ms 10.3 ms 0, 1, 2, 3
0.6 2.0 ms 7.6 ms 0, 1, 2, 3
0.5 1.4 ms 3.6 ms 0, 1, 2, 3
0.35 0.95 ms 4.0 ms 0, 1, 2
0.25 1.1 ms 4.1 ms 0, 1, 2

0.5 and 0.25 are the cheapest per pixel (whole-pixel averaging); other factors use an exact area resize that costs a little more but is never slower than not scaling (before 0.6.2 they were).

The price of scaling is range: a level must still be ~45 px wide after scaling, so the smallest inner levels (and markers far away) drop out first. Good fit: 4K cameras watching a marker that fills a decent part of the frame.

import nanofractal as nf

p = nf.DetectorParams()
p.detection_scale = 0.5
det = nf.FractalDetector("FRACTAL_5L_6", marker_size=0.20, params=p)

Threads

nf.set_num_threads(n) is process-wide and caps how many cores one detect() uses. Speed-up of a 1080p frame on 6 physical cores:

threads 1 2 4 6 12
ms 3.2 1.8 1.4 1.1 1.1–1.4

Gains stop around the number of physical cores. detect_batch is not affected: it always runs one frame per worker and never splits frames.

Under load: fewer threads can be faster

The table above is an idle machine. When the same process (or the system) also captures, encodes or previews video, detect()'s threads compete with that work, and the extra synchronisation can cost more than it gains. A Raspberry Pi 5 report (camera capture at 60 fps + preview, 4 cores):

set_num_threads 2 4
frames/s processed 57.5 32.5

On a small board memory, not the cores, limits larger frames. Raspberry Pi 5 (0.7.0, idle, clutter scene, ms per detect):

frame 1 thread 2 threads 3 threads 4 threads
640×480 0.67 0.36 0.26 0.21
960×720 1.59 0.94 0.84 0.90
1280×720 2.41 1.65 1.52 1.66
1456×1088 4.70 3.65 3.56 3.84
1920×1080 6.73 5.35 5.00 5.58

From about 960×720 up a third and fourth thread add little, and with other work on the board (here: a package update in the background) four threads were slower than one. detection_scale=0.5 cuts 1280×720 to 0.72 ms on one thread — on such boards it is the bigger lever. Measure your board with python benchmarks/pi_threads.py.

Start with half the cores, then measure the whole pipeline (capture → detect → pose), not detect() alone:

import os
nf.set_num_threads(max(1, (os.cpu_count() or 2) // 2))

Cheap things that matter

  • Create detectors once, outside the frame loop (creation is cheap, but the first frames also warm up internal buffers that are then reused).
  • Keep frames C-contiguous uint8; strided views raise rather than copy.
  • Don't request inner points on frames where you don't use the pose.
  • Gate poses on reproj_err instead of lowering detection thresholds.

Measure on your machine

python -m nanofractal.bench --resolution 1920x1080 --scene perspective --mode detect
python -m nanofractal.bench --resolution 3840x2160 --mode scale05

--scene clean|perspective|clutter, --mode detect|inner|scale05. The repository's benchmarks/fractal_bench.py runs a full matrix and also reports detect_batch throughput.