Tree instance separation — implementation plan

2026-08-15 · verticalsigns repo · baseline splitter for detected tree clusters

Summary

Status 2026-08-15, end of day — baseline SHIPPED (commit 85f97f1, producer tree-instances-0.3.1). Four iterations, each validated on the same 24 A1B0 segments on battlebox with per-render vision audits: bad verdicts 50 → 30 → 19; final run: 164 clusters → 223 instances + 5 hedges, abstain 0.81 (dominated by two honest megacluster/leak refusals). The 238k-pt flagship blob now splits into ~13 coherent crowns. Remaining known-hard: sparse far-range canopy (abstains as partial), band-vs-shrub-row ambiguity (needs gold set). Next: recall rescue track (Phase R) — ~480 RF-rejected green clusters await routing through this splitter.

Add a post-detection tree instance splitter to the verticalsigns repo: it takes the tree clusters the existing classical tree path emits (chroma-filtered vegetation blobs with per-point masks) and splits merged blobs into individual trees, assigning a per-point tree_instance_id with explicit abstention (−1) where the geometry can't support a split.

Algorithm is the research-backed hybrid geometric baseline (no training data): stem-seed detection (DBSCAN on an adaptive below-crown height slice + circle/verticality evidence) → region-growing crown assignment to the nearest compatible stem → abstention for seedless canopy regions and hedge-like blobs. Implementation runs as a dynamic multi-agent workflow (cheaper models for mechanical work), verified on Abschnitt 1 data on battlebox with rendered per-instance colorings inspected visually.

Key decisions

Decision	Choice	Why
Code home	Inside `3dai.iolabs.pointcloud.verticalsigns`, new module `tree_instances.py` (+ `_config_treeinstance.py`)	User decision. The tree path (`trees.py`, chroma filter, vegetation RF) lives here and produces the input clusters. Not in mlsegmentation — that model isn't trained yet (not enough annotated data); this splitter works on this repo's detections.
Input contract	Detected tree instances + per-point membership from `point_masks.npz` (shared `iolabs.common.point_masks_io` sidecar), points loaded from the segment's `*_run3_points.npz` records	Verticalsigns already writes the sidecar (`detect.py:_dump_point_masks`, spec-constrained `instance_type` incl. `tree`). Verified on battlebox: A1B0 run `signs2_A1B0` has tree masks incl. a 238k-point merged blob (47×38 m). Where masks are missing, enable/reuse the same sidecar code — no new format.
Algorithm	Stem-seeded geometric split (treeX/Point2Tree lineage): adaptive stem-band slice → 2-pass DBSCAN → stem evidence score (verticality, circle fit, height reach) → region-growing / nearest-compatible-stem crown assignment	Deep-research verdict (2026-08-15): only validated learning-free route; parity with DL on ground-based clouds. Roadside trees here are short (3–7 m, crown base ~2 m), so the literature's fixed 1–4 m band becomes an adaptive below-crown band per cluster.
Abstention	Mandatory: `tree_instance_id = −1` + machine-readable reason (`no_stem_seed`, `hedge_like`, `low_evidence`). Whole-cluster `split_quality` enum: `single/merged/partial/hedge/non-tree/uncertain`	Research shows one-sided MLS is geometrically underdetermined at occlusion gaps and interlocked crowns; guessing produces silent undersegmentation (61% vs 16% over-). Seed-evidence test first — simplest defensible rule.
Output artifact	Per-segment `tree_instances.npz` sidecar next to `point_masks.npz`: `record_id/point_index` (same identity scheme), `tree_instance_id int32` (−1 unknown), per-instance `instance_confidence float32[K]`, `split_quality`, `abstain_reason`, `producer_version`	Keeps the detection ontology untouched; instance IDs live in a separate versioned artifact keyed on the same stable point identity (record + point index) so results are evaluable on raw source points.
Validation data	Abschnitt 1 (branch 0), chroma filter enabled (`prod2_A1B0.config.json` semantics), on battlebox (RTX 5070 Ti box, data at `/mnt/d/a123-data`, existing run at `~/seg3d_out/signs2_A1B0`)	User decision. Existing A1B0 outputs already contain tree point masks → the splitter can run on them immediately, no full detector re-run needed first.
Execution model	Dynamic Workflow: design/authoring stays with the orchestrator, implementation + tests fan out to cheaper agents (Opus/Sonnet tiers), final review by the orchestrator	User instruction: preserve Fable credits where its intelligence isn't necessary.
Licensing	Pure own implementation; treeX/Point2Tree are read-only references (Point2Tree is GPL-3.0 — no code vendored)	License hygiene per handoff.

Phases

Phase 1 — Module design & artifact contract small

New module src/iolabs_point_cloud_detection_verticalsigns/tree_instances.py: pure function core split_tree_cluster(points_xyz, rgb, z_ground, config) → TreeSplitResult (instance labels, per-instance confidence, split_quality, abstain reasons). No I/O in the core — testable on synthetic clouds.
_config_treeinstance.py following the repo's config-dataclass pattern (_config_*.py + JSON section, like chroma_vegetation); enabled via a tree_instances section in the run config, default off.
Thin I/O shell: read a segment's point_masks.npz (tree instances only) + record *_run3_points.npz, write tree_instances.npz sidecar + a JSON summary (per-cluster seed count, abstention population size — the early health metric).
Standalone CLI driver script (scripts/) so the splitter can run over an existing detector output dir without re-running detection.

Phase 2 — Splitter algorithm, test-first core

Height normalization: per-point height above ground from the detection's z_ground (flat-ground assumption within a cluster footprint; fall back to local ground percentile for large blobs).
Adaptive stem band: per cluster, band = [0.5 m, min(4 m, crown_base)] where crown_base comes from the vertical density profile (foliage onset); guards for short trees (3–7 m observed on A1).
Stem seeds: 2D DBSCAN on stem-band points (xy), then per-candidate evidence score: vertical extent within band, horizontal compactness (~stem radius via circle fit / RANSAC-lite), verticality of the local axis, greenness (stems are less green than foliage — chroma helps here). Reject non-stem clutter (fence posts already filtered upstream by type).
Crown assignment: region growing on a voxel/k-NN graph from seeds upward+outward; unclaimed canopy points go to nearest compatible seed by 3D graph distance with a max-gap constraint (Point2Tree's cumulative-gap idea, own implementation); points unreachable under the gap constraint → abstain.
Cluster verdicts: 0 seeds → whole cluster abstains (no_stem_seed or hedge_like if aspect ratio / height profile says hedge); 1 seed → single (identity split); N seeds → split with per-instance confidence from seed evidence × assignment margin.
Tests (pytest, repo conventions): synthetic scenes — two cones with touching crowns (must split), hedge slab (must abstain), single tree (must not split), stemless canopy ball (must abstain), N-tree row with known ground truth (matched-instance precision/recall assertions).

Phase 3 — Battlebox run on A1 medium

Sync the branch to battlebox (~/dev/3dai.iolabs.pointcloud.verticalsigns, own worktree/branch; uv available), run the splitter CLI over ~/seg3d_out/signs2_A1B0 tree masks (all segments with trees; includes the 238k-point merged blob in segment_014).
Collect the summary stats: trees in, instances out, seeds per cluster, abstention population share, runtime. The abstention share is the early kill signal — if >80% of merged-blob area abstains, the stem-evidence route is starved and we revisit before polishing.
If the existing signs2_A1B0 run turns out to lack the chroma filter or point masks for some segments, re-run detection for a handful of representative segments with prod2_A1B0.config.json + dump_point_masks enabled rather than the whole corridor.

Phase 4 — Visual verification gate

Render per-segment PNGs on battlebox (matplotlib headless, same approach as the existing vegnotch/pc_render.py tooling): top-down + oblique views, one color per tree_instance_id, gray = abstained, over an RGB context backdrop.
Pull renders back and inspect them visually (orchestrator vision pass): checklist per scene — plausible tree count, no obvious crown split down the middle, hedges gray not rainbow, isolated trees single-colored, conifers present (2026-07-17 regression: conifers must not vanish).
Iterate parameters (band, DBSCAN eps/min_pts, gap constraint) on the worst 3–5 scenes; re-render; stop when visual verdicts stabilize.
Deliverable: short results section + gallery of before/after renders, plus the stats table.

Phase R — Recall rescue track (user: "a lot of trees were undetected — recall here is a huge win") next after baseline

Measured on the A1B0 run: the tree path emitted only 233 trees while rejecting 1,315 clusters via reject_rf_below_threshold; ~480 of the rejects are green (ExG ≥ 0.10), substantial (≥500 pts), ≥2 m tall and non-retroreflective. Visual sampling confirms they are real vegetation: scrub walls, embankment bush fields, sapling stands (with visible stems), incl. 27k–77k-point blobs. Potential ≈3× detector-level recall gain.

Rescue path in trees.py: config-gated post-pass — RF-rejected clusters passing a chroma-vegetation evidence gate (ExG, size, height, low hi-intensity, bridge guard) are routed through the new splitter core: stem seeds found → emit per-instance tree detections (reason chroma_stem_rescue); hedge verdict → emit the new hedge feature class; neither → stay rejected. Stem evidence keeps precision honest — green alone (grass) never suffices.
Chroma × TCS adjustment (user): revisit the chroma thresholds (chroma_veg_exg_min 0.155 vs observed real-vegetation rejects at 0.15–0.29) together with the TCS ground filter interaction (embankment scrub: TCS may eat low vegetation or bias height-above-ground under dense cover). Tune on the reject population, validate visually.
Hedge in the point-mask enum: shared point_masks_spec.md constrains instance_type to sign/delineator/pole_other/gantry_or_gate/tree — adding hedge is a small cross-repo change in iolabs.common (+ 3dsegmentation class table, proposed las_code 4). Until then hedges surface in tree_instances.json only.
Measurement: before/after emitted-vegetation counts per segment + visual audit of a rescued sample; no gold set yet, so gains are reported as counted-and-eyeballed, not as recall metrics.

Phase 5 — Later (explicitly out of this pass)

Gold set annotation (corridor-held-out) + matched-instance metrics (precision/recall/F1, matched IoU, under/over-segmentation counts) — needed before trusting any numbers.
TreeLearn zero-shot challenger comparison.
Range × occlusion abstention refinement (current pass uses seed-evidence only).
Wiring the splitter into the production detector run (config-gated) + Nexus release via wrap-up.

Risks

high Short roadside trees break the 1–4 m literature band. A1 trees are 3–7 m with crown base ≈2 m; half the points of real blobs sit in the naive band. Mitigated by the adaptive below-crown band; residual risk that crown-base estimation fails on dense hedges → they abstain (acceptable).
high Giant mixed blobs (hedge + trees interlocked). The 47×38 m segment_014 blob likely mixes hedge rows and trees; partial splitting with honest partial/hedge quality labels is the target, not full separation. Visual gate decides if partial output is usable.
med No quantitative ground truth this pass. Visual checks catch gross errors only; per-point accuracy on touching crowns is unquantified in literature too. Phase 5 gold set is the only fix; until then results are labeled qualitative.
med z_ground flatness assumption. Large blobs on embankments have sloping ground; a single z_ground skews the stem band. Local ground fallback planned; if insufficient, reuse the ground model from the detector run.
low Battlebox environment drift. Repo venv (py3.12) exists and loads the data already (verified live); worst case is a uv sync.

Open questions

Multi-stem trees & continuous hedges ontology: is a coppiced multi-stem clump one instance or N? Current plan: one instance per detected stem, with split_quality=uncertain when stems are <1 m apart; hedges abstain wholesale. Needs your ruling before any gold annotation (Phase 5), not blocking this pass.
Where should instance IDs surface downstream (XML export? LAS classification codes?) — sidecar-only for now; export wiring deferred until the baseline earns trust.
Chroma availability on all A1 records: chroma features degrade to no-ops on achromatic input by design; segments with dead RGB will fall back to geometry-only seeds — acceptable, but worth counting in the run summary.