Vertical signs — pass 3: vehicles, out-of-domain vetoes, and the A1 breakthrough
Summary
Pass 3 shipped three defaults, each adversarially judged and then verified by full re-runs: a single-record transient veto (half of A4_5's accepted signs were moving vehicles), a relaxed delineator footprint 0.65 → 0.85 m (A1 posts merge with verge grass), and a corridor-gated ML veto (the RF verifier is measurably out-of-domain on run4-less data). Net effect: A4_5 sign precision 33% → 67% at −6 confirmed-vehicle FPs and +3 judged-real delineators (111 → 114); the A1 eval slice went 10 → 59 detections with ~34 of the +50 confirmed real by two independent judges. The pass-2 mystery "sign CR agreement 0.000" is resolved: it was a structural artifact of single-record objects, not detector noise.
Half of the accepted A4_5 signs were moving vehicles
A visual audit of all 12 accepted signs (3-panel crops: topdown + two side elevations) found
6 vehicles, 4 real signs, 2 vegetation cones. Every vehicle shares one signature: it was seen by
exactly one driving pass (n_records_present=1) — a transient exists in one
record by physics — and is panel-like (verticality ≤ 0.07, 2.9–5.4 m long, under 2 m tall). Static
inventory is re-observed: 95% of accepted delineators have 2+ records.
bright_panel rule saw only a large retroreflective slab.
The veto is a conjunction — single-record and vehicle-shaped — because coverage alone is not proof: the seg-134 gore beacon is a real single-record sign (only one pass covers the gore) and survives on its verticality 0.9999. Full-sweep verification removed exactly the six judged vehicles and nothing else; on A1 it also caught a 4.5 m single-record slab. Research memo confirms the mechanism is established practice (repeated-traversal persistence, Schauer & Nüchter's Peopleremover).
The sign capture–recapture verdict was an artifact
Pass 2 reported sign split-half agreement 0.000 and treated it as the top defect. Tracing each full-run sign through both CR halves shows why it could never be otherwise: 58% of accepted signs are single-record objects — when records are split into halves, such an object's points land entirely in one half, so it cannot be matched. Six of the seven single-record "signs" were the vehicles above; the artifact and the FP family are the same physics. Delineator CR is essentially unaffected (5% single-record), so the pass-2 delineator agreement gains stand.
Consequence: CR stays the QA instrument for delineators; sign-path QA needs either multi-record-restricted CR (only ~5 objects on this slice — too few) or judged audits.
A1 transfer diagnosis: gates, not radiometrics
The intensity census (208 segments, full uint16 histograms) killed the calibration hypothesis: A1 is globally brighter (bulk p50 ≈ 37k vs 29k, 24% vs 8.5% of points above 40k) but the bright-vs-bulk contrast survives, and per-scan-angle analysis shows the bulk is flat across incidence. The real blockers, from a judged audit of 70 stratified near-miss rejects (672-candidate pool):
| Blocker | Evidence | Fix |
|---|---|---|
| relaxed-footprint cap 0.65 m | 8 judged-real posts fail ONLY this conjunct: post+grass merged clusters at footprint 0.67–0.83 with verticality ~0.99 | cap → 0.85 (shipped) |
| RF veto out-of-domain | all 5 judged-real seg-048 signs (X-crossbuck family) score P 0.51–0.59 vs the 0.62 threshold anchored on A4_5 training reals; the dist-clamp was tested and is innocent | ml_veto_requires_corridor (shipped) |
| min_points 300 / sparse strata | judged sparse family: a few real posts at 45–134 points, but Opus/Grok disagree heavily there | deferred (lattice-support rule, pass 4) |
Shipped defaults (commits 175370f, 5bd7443 — local only)
- shipped
single_record_transient_veto=true— nrec=1 ∧ verticality≤0.3 ∧ len≥2.0 ∧ h≤2.5 → reject accepted-sign verdicts assingle_record_transient. - shipped
delineator_relaxed_footprint_m: 0.65 → 0.85— A4_5 cost measured first: +3 judged-real, 0 lost. Real (0.66–0.83) and FP (0.68–0.85) footprints fully overlap, so no tighter cap separates; the veg leak is a texture problem (task #14). - shipped
ml_veto_requires_corridor=true— segments without run4 road-surface files score-and-annotate (ml_confidencepreserved) but do not veto. A4_5 byte-identical. - accepted cost the A1 delta contains 9 judged-FP detections (6 vegetation, 3 stakes/barrier) and 7 disputed. Judged precision of the delta ≈ 79%.
Verification: full default-config re-runs of both datasets are detection-identical to the judged experiment runs (A4_5: 143 segs, sym-diff 0; A1: 65 segs, sym-diff 0). 247 unit tests pass; every flag off restores the byte-identical pre-pass-3 detector.
Adversarial judging
Two rounds, refute-framed ("presume FP until the pixels prove otherwise"), Opus + Grok independently on
every candidate, Fable tiebreaks on splits. Round 1: 70 stratified A1 near-misses → 17 high-consensus real
misses, guardrails/vegetation/noise correctly rejected elsewhere. Round 2: all 36 unjudged new detections of
the relaxed config → 26 upheld, 10 refuted. Verdicts in out_eval/p3_census/*.csv (kept out of git).
Research memo (Codex, 36 web searches)
out_eval/research_pass3_memo.md. Highlights: (1) per-pass persistence filtering is established
(Peopleremover, persistence priors); use it only as a rejection gate, never to promote — matches the shipped veto.
(2) For out-of-domain verifiers without target labels: abstention via source-support (kNN density on the RF's
feature space) is the cheap upgrade over the corridor proxy. (3) Post-from-grass separation: bright-core
resegmentation / cylinder-RANSAC on intensity-filtered subsets. (4) Tree growth shelters are a documented
retroreflective-plastic FP source; multiscale planarity + panel–stem topology separates plates from cones.
Pass-4 queue (ranked)
- Static-FP texture cues (task #14, now the top defect): multi-radius plate regularity / multiscale planarity against the 9 judged-veg FPs (A1 scrub band + A4_5 seg-038 cones), per research memo §4.
- Bright-core resegmentation for merged post+grass clusters (memo §3) — could replace the footprint relaxation with something principled and reclaim the disputed 0.80–0.85 band.
- Sparse-delineator recovery via lattice support: accept sub-300-point bright posts only when they continue a 25/50 m chain (StVO prior); judges split on this family, so instrument first.
- RF verifier refresh with weak supervision: 33 consensus CR pairs + ~150 judged verdicts accumulated across passes; source-support abstention (memo §2) as interim.
- rail_reflector class (carried from pass 2): rail-coordinate peak detection, segs 095–097 chain.
Judge panel still not convened: no stop condition is plausibly true — improvements continue without human annotation, and the pass-3 delta was largely rule-level, not ML-bound.