Plan 2 — `greenness_exg`: A1-only chromaticity lever

2026-07-17 · rev 3 (rewritten for the implementation agent) · Jira AI3D-339 (commit prefix AI3D-339:) · sibling of plan 1 (TCS + conifer rules) · all compute on battlebox

Summary

Bug. Cluster greenness is normalized by a segment-wide RGB max (features.py:211-212, grid.py:140-145): one retroreflective sign anywhere in the segment saturates the denominator and drives a dark spruce's greenness toward 0.

Fix. Per-point chromaticity, no cross-cluster coupling:

exg = (2g − r − b) / (r + g + b + 1e-6)     # range [−1, 2]

Two new cluster fields: greenness_exg (median), greenness_exg_iqr (spread — foliage is chromatically heterogeneous, painted steel is not). Strictly additive; old greenness stays byte-identical.

Why a rule must consume it. Both shipped RF bundles are locked to their own feature lists (ml.py:437-441 getattr loop); retraining is out of scope. A new field is invisible to the models — phase 5 (the consuming rule) is the crux; everything before it is plumbing.

Scope + exit. A1 only (the sole live-colour dataset). Config-gated, default-off. Kill without ceremony if A1 distributions don't separate (phase 7).

Contract (binding decisions)

Additive only — never touch greenness. It feeds two shipped RFs (ml.py:56, :102); changing its semantics sends them out-of-distribution values. A config flag is no protection — the flag is what changes the value the RF receives. (Settled fact, not a question: the shipped veg bundle's greenness importance is exactly 0.0000 → it was trained on colour-free data → the slot is dead weight, nothing is broken, and a future retrain with real chroma would be a feature, not a fix.)
Compute ExG in the reader, one float32 per candidate. R and B live only inside accumulate_candidates (loaded grid.py:132-134, freed :157/:210-225); cluster membership is decided later (DBSCAN :381-464), so a per-candidate array candidate_exg is the minimum. The numerator (green_excess) already exists at :160-167.
The consuming rule is a NEW, gated, default-off sign-path reject (R2) — not a repoint of the crown gate. Repointing classify.py:91 is verdict-invariant (it only picks which reject label a crowned cluster gets — both labels promote identically). The target FP — a dark conifer with crown_area ≤ 4 m² or isotropy < 0.75 — never reaches the crown gate; it leaks into pole_other/sign_post. Only a new reject catches it.

A1 only. Colour by dataset (value-verified):

dataset	colour
251017_Color_Abschnitt_1_long (825) / _short (691)	live — hundreds of distinct values per channel
260416_Abschnitt_4_5 (1508)	sentinel white — constant 65535 all channels
260605_Abschnitt_3 (×3, 696 ea)	treat as intensity-only — coloured and colour-less scans overlap spatially; cluster medians blend real chroma with filler into plausible-looking noise
Abschnitt 2	sentinel white

On sentinel data green_excess = 0 → exg ≡ 0, iqr ≡ 0 → the field is self-neutralizing and the R2 gate can never fire there. Safe to compute unconditionally.

Validate NPZ fields on VALUES per dataset, never key presence (three known modes: absent — return_number; constant — scan_angle in A3, RGB in A2/A4_5; mixed — RGB in A3). Phase 0 is this gate.
No labels → label-free verification only. No recall/precision/AUC anywhere. Use stratified distributions over existing type/reason verdicts (a weak proxy, stated openly) + a visual overlay pass. The agent does that pass itself and fixes what it sees — view the --dump-point-masks PNGs directly, or open the cloud in CloudCompare on battlebox (Codex computer-use, optional) for ambiguous clusters; human review is escalation only, never first.
New config keys; never reuse tree_greenness_hint = 0.45 (config.py:184) — it is calibrated to the broken normalizer; ExG foliage is typically ~0.05–0.4. New: chroma_veg_enabled (default False), chroma_veg_exg_min, chroma_veg_exg_iqr_min; values come from phase 6.
Battlebox mechanics: ssh battlebox "bash -s" <<'EOF' … EOF; base python has no numpy — repo venv or uv run; /mnt/d is slow 9p — intermediates to local ext4. Data: /mnt/d/Data/02_AI 3D modeling/00_data/251017_Color_Abschnitt_1_{long,short}/lane_points/.

Phases

Phase 0 — per-dataset value-validation gate S

~30 NPZ per A1 half: per-file min/max/unique for r,g,b + per-point ExG histograms on off-ground points → confirm A1 colour is per-point usable (not banded, not per-scan-constant).
Cross-scan exposure probe: per-file median ExG over road-surface points — asphalt should be ~0 everywhere. Scatter beyond ~±0.05 → a global threshold is shaky (feeds R5 and the kill decision).
Spot-confirm A4_5/A2 sentinel (exg exactly 0) and A3 contamination — record, exclude, don't "fix".

Done when: one-page stats dump; proceed only if A1 passes liveness and exposure scatter is bounded.

Phase 1 — TDD the ExG math S

New tests/test_features.py, tests first:

pure green (0, 255, 0)        → 2.0
grey / white (v, v, v), v>0   → 0.0   # incl. sentinel 65535³
pure red (255, 0, 0)          → −1.0
empty / all-zero input        → median 0.0, iqr 0.0 (no NaN)

Also: median/IQR of a mixed foliage-like sample; float32 input; ε keeps black points 0-division-safe.
IQR via np.percentile two-liner — features.py imports no scipy; keep it that way.
The sentinel-neutrality case is load-bearing (see contract #4).

Done when: uv run pytest tests/test_features.py red → green (pure numpy, no data).

Phase 2 — reader: candidate_exg in grid.py M

At :160-167, next to selected_green: selected_exg = (2·selected_green) / (rgb_sum[candidate] + 1e-6), float32; rgb_sum freed with r,g,b.
Thread: new exg_chunks list (:121), append (:193-201), concat + empty-segment arm (:227-246 — keep the M=0 point-masks contract), return tuple 15 → 16 (:274-290), docstring (:84-103).
Call-site unpacks: detect.py:475-491; per-cluster indexing at :526, re-featurize at :604; tree-path pass-through detect.py:708-710 → trees.py:58-63, :111-113.
segment_rgb_max and the old greenness path: byte-identical.

Done when: suite green; one A1 + one A4_5 segment on battlebox — A4_5 outputs byte-identical except the new all-0 column.

Phase 3 — features: additive fields S

ClusterFeatures (features.py:15-59): append greenness_exg: float = 0.0, greenness_exg_iqr: float = 0.0 (after the defaulted tail at :37).
compute_cluster_features (:180-190): new green_exg parameter; median + percentile-diff IQR next to :211-212 (empty/degenerate → 0.0). Old greenness line untouched.
Call sites: detect.py:527, :600, trees.py:107.

Done when: phase-1 tests extended to compute_cluster_features; full suite green.

Phase 4 — clusters.csv seam S

Append both fields to CSV_FIELDS (detect.py:54-85) and the hand-maintained dict in _features_to_row (:152-189, next to greenness at :174) — phase 6 reads this CSV.
Comment at the feature-name lists: adding greenness_exg to a list in a future retrain hard-KeyErrors (ml.py:149-152) on every pre-existing clusters.csv.

Done when: one segment run carries both columns; verticalsigns-train corpus loading (dry run) parses old + new CSVs.

Phase 5 — the consuming rule (the crux) M

rule	what	verdict impact	call
R1 — repoint crown-gate label (`classify.py:91`)	swaps which reject label a crowned cluster gets	none (verdict-invariant, contract #3)	skip
R2 — new sign-path chroma-vegetation reject	in `_reject_reason` after the crown gate (`classify.py:93`), gated on `chroma_veg_enabled`: reject as `"chroma_vegetation"` when `exg ≥ min` ∧ `iqr ≥ min` ∧ volumetric evidence (`change_of_curvature` high or `plate_thickness_m` over limit) ∧ `hi_intensity_fraction` low. Chroma alone never vetoes a bright plate — the multi-conjunct shape is load-bearing (literature-validated ≥2-cue rule), do not relax it. Add reason to `TREE_REJECT_REASONS` (`classify.py:15-17`); mirror in `ml.py` `VEG_REASONS` (`:117-119`)	removes dark conifers/bushes that dodge the crown gate and leak into `pole_other`/`sign_post`	primary
R3 — tree-path rescue (`trees.py:147-150`)	gated: emit when RF confidence within a margin below threshold ∧ `exg ≥ hint` (reason `vegetation_rf_exg_rescue`)	tree emission only; zero sign-path exposure	optional follow-up

Config: chroma_veg_enabled: bool = False + two thresholds in DetectorConfig, JSON loader per the tree.get(…) pattern (config.py:613-616), plus an A1-only override JSON. Default-off ⇒ every run without the override is bit-identical to today.
Thresholds come from phase 6; placeholders in code are fine, the override ships only after phase 6.
TDD (tests/test_classify.py): dark-conifer features rejected with flag on; identical verdict with flag off; bright plate never chroma-vetoed; sentinel (exg=0) never fires.

Done when: suite green; an A4_5 segment with the flag on is byte-identical (proves data-safety, not just config-safety).

Phase 6 — label-free validation on A1 (battlebox) M/L

Detector over a broad A1 sample (both halves; intermediates on ext4), rule off, collect every clusters.csv.
Distribution study: greenness vs greenness_exg (+ IQR), stratified by existing verdicts — accepted sign/delineator/sign_post/pole_other vs rejected tree_crown_*/bush_like_core/forest_context. Percentile tables + histograms per stratum.
Threshold pick: chroma_veg_exg_min > upper tail of accepted-steel strata (e.g. P99 of accepted signs/delineators); chroma_veg_exg_iqr_min from the foliage-vs-steel IQR gap. Documented.
Rule on/off diff (agent-first visual pass): re-run with the A1 override; diff detections.json; the agent views every flipped cluster itself — --dump-point-masks PNGs read directly, or CloudCompare on battlebox (Codex computer-use, optional) for ambiguous ones — and fixes what it sees (tighten thresholds, adjust the guard) before escalating anything. Hunt specifically for old/faded signs among the flips — deteriorated retro-sheeting sits at ~0.45 normalized intensity, below typical brightness guards, and a moss-tinged one is R2's most plausible real-marker victim; a flipped faded sign is a fix trigger, not a note-and-move-on.
Log: does A1 contain conifers at all; does phase-0's exposure scatter hold corpus-wide (R5).

Done when: distribution report + before/after diff with per-cluster visual verdicts. No metric claims.

Phase 7 — kill-or-keep S

Kill if: exg strata overlap like greenness strata; or exposure scatter swamps the foliage/steel gap (per-segment normalization is out of scope — wide scatter IS the kill line, not a patch opportunity); or the diff removes real markers. Revert phase 5's rule; keep phases 2-4's plumbing only if plan 1's conifer work wants it — otherwise revert too.
Keep if separation is visible and the diff is clean: land rule + A1 override, cross-link into plan 1.
Comparison bar: judge R2 against the corpus-wide geometry-only texture cue (scattering / sphere-cylinder ratio — plan 1 Phase-3b option), not against nothing. Keep chroma only if A1 evidence shows it beats or complements that.
Verdict + phase-6 evidence goes to Miro under AI3D-339; the plan pre-commits only to honouring the kill rule.

Risks

	risk	mitigation
risk	No regression corpus — the 13 curated hard segments are all colourless A4_5	default-off gating + A4_5 byte-identity checks (phases 2/5)
risk	R2 eats real markers (green-tinged, moss-covered, faded, foliage-occluded posts)	brightness guard in the rule; default-off; A1-only override; agent views every flipped cluster (incl. the faded-sign hunt) and fixes what it sees before human escalation
warn	Inert without retrain — if R2 is killed, phases 2-4 are dead weight	phase 7 reverts dead weight on kill
warn	A1 may have few/no conifers	phase 6 answers it; the rule may earn its keep on deciduous/bush FPs — kill rule applies either way
warn	Cross-scan white-balance drift would make a global threshold meaningless	phase 0 asphalt probe + phase 6 corpus check
warn	Threshold-constant confusion — `tree_greenness_hint=0.45` is on the wrong scale for ExG	new keys only; reviewer checklist item
warn	Plumbing churn vs plan 1 — tuple extension + `compute_cluster_features` signature touch plan 1's seams	plan 1 merges first; keep this branch rebased
warn	Future-retrain KeyError seam (`ml.py:152`)	comment at the feature lists (phase 4); not this plan's problem

Open questions (for Miro — do not block phases 0-4 on this)

Resolved since rev 2 and removed: rule choice = R2 primary / R3 optional / R1 skipped (decided in phase 5); conifer count in A1 (phase 6 answers it empirically); the veg bundle's training corpus (settled — greenness importance 0.0000 ⇒ trained colour-free, slot dead, nothing broken); the keep bar (phase 7's comparison against the geometry-only texture cue); wide exposure scatter = kill line (folded into phase 7).

Q1 — is A1 in shipping scope, or exploration-only? If exploration-only, this whole lever is R&D and the phase-7 "keep" bar rises. Not answerable from the repo.