AI3D-337 · Slope-aware SMRF ground gate

2026-07-17 · Miro / implementation review · branch worktree-ai3d-337-slope-handling off main@d612f67

Summary

Tablecloth's SMRF ground separation has only one of canonical SMRF's two slope mechanisms: the morphology loop scales with slope (dh_max = slope_threshold · radius), but the final point gate is a slope-blind absolute threshold z − surface_z ≤ max_elev_diff_m (0.05 m). On grades that matter (≥15%), genuine pavement gets cut. This plan adds a PDAL-style gradient-scaled classification threshold (elev_scalar, default 0.0 = bit-exact current behavior) plus a CSF slope-smoothing passthrough (csf_slope_smooth), with steep/cross-slope/crowned/embankment fixtures and streaming-parity tests. Execution is a multi-model workflow: Grok writes tests (RED), GPT-5.6 Sol implements (GREEN), Opus + Sonnet + Grok + Sol review in parallel, Grok fixes, Fable orchestrates and ships a draft PR.

Source research: 48-agent deep-research run (2026-07-17), vault note "Tablecloth slope-handling deep research"; handoff at /tmp/handoff-2eYFv2.md. No re-research needed.

Key decisions

DecisionChoiceWhy
Slope mechanism Option A — per-point threshold max_elev_diff_m + elev_scalar · |∇surface|, gradient bilinear-sampled at each point Literature-standard (Pingel 2013 / PDAL SMRF, PDAL default scalar 1.25); O(grid) memory; research-verified as the right fix
Default value elev_scalar = 0.0, bit-exact legacy path (the 0.0 branch short-circuits to the exact current expression) Safe rollout; default-on only after A/B on real segments
Where the gradient lives New grad grid field on frozen SmrfSurface, computed in _finalize_smrf_surface after the opening loop In-memory and streaming both flow through _finalize_smrf_surface + classify_against_surface → streaming parity holds by construction, no forked paths
CSF quick win New csf_slope_smooth: bool = true wired to params.bSloopSmooth (exact upstream typo), hasattr-guarded like interations Upstream C++ default is true, so default matches today's untouched behavior; CSF extra isn't installed locally, so upstream default is the best available reference
Option B (morphology loop changes) Deferred — only as a contingency if A cannot pass the steep fixtures Handoff explicitly says don't do it preemptively; slope_threshold=0.15 is tuned for curb-vs-wall on flat ground
Process Strict TDD: fixtures + failing tests first, watched to fail for missing-feature reasons, then implementation The steep-fixture failure mode needs empirical confirmation (see uphill-edge analysis below); tests are the spec
Execution model One Workflow run: Grok 4.5 (cursor) → GPT-5.6 Sol (codex) → Opus/Sonnet/Grok/Sol review panel → Grok fix; Claude fallbacks per stage; Fable orchestrates, verifies, commits, opens draft PR Requested multi-model split (incl. Sol as extra reviewer, per Miro); Grok tokens free today; nobody but the orchestrator commits

Pre-computed failure analysis (head start for GREEN)

On grades g above slope_threshold (0.15), grey opening of a ramp near the uphill grid edge returns ≈ f(edge − window) (dilation can't reach uphill past the boundary), so (surface − opened) > slope_threshold·radius triggers there and the replaced band (last ~2–3 cells) ends up to g·window below true ground and nearly flat. The final surface gradient inside that band is ≈ 0, so gradient scaling alone may not rescue it — the GREEN agent gets this analysis and, if confirmed, is allowed one minimal principled adjustment (gradient propagation into replaced bands, or an uphill-edge replacement guard) that keeps every existing test green, including crop-edge object removal. Away from edges the residual misfit is per-cell discretization ≈ g·cell_m (0.036 m at 18%) vs the 0.05 m gate.

Phases

Phase 1 — RED: fixtures + failing tests Grok 4.5

Tests only, no src/ changes. Baseline suite must be green first.

  • Fixtures (tests/fixtures/synthetic.py): extend sloped_plane_lip_vehicle with backward-compatible grade_y (cross-slope); add crowned_road_vehicle (center-ridge crown + vehicle blob); add grade_break_embankment (flat road, then 30–40% bank).
  • Steep break test: 18% grade, default config → plane-kept bound documenting the defect (calibrated to observed behavior; raise grade toward 0.20–0.25 only if 0.18 doesn't break).
  • Fix tests with elev_scalar=1.25: steep (plane ≥ 0.99 excl. blob halo, blob removed ≥ 0.95, lip kept ≥ 0.95), cross-slope, crowned road, embankment (window must not flatten the grade break).
  • Lip-keep assertion on the existing 6% fixture under defaults (pins currently-unasserted behavior; expected to pass today).
  • Bit-exact regression: np.array_equal between default config, explicit elev_scalar=0.0, and a manually computed fixed-gate reference mask.
  • Streaming parity on the steep fixture with elev_scalar=1.25 (chunked vs in-memory, exact mask equality).
  • Config coercion tests for both new keys; CSF-behavior tests gated by importorskip("CSF").

Verify-RED: every new test must fail for a missing-feature reason (unknown config key / missing parameter / demonstrated defect) — not a collection error.

Phase 2 — GREEN: implementation GPT-5.6 Sol
  • config.py: elev_scalar: float = 0.0 (SMRF block), csf_slope_smooth: bool = True (CSF block); mirrored in tablecloth.default.json (parity is test-enforced).
  • ground.py: SmrfSurface.grad; gradient via np.gradient(surface, cell) + np.hypot (size-1-axis guard); classify_against_surface(..., elev_scalar=0.0) with exact legacy expression on the 0.0 branch; classify_mask_from_chunks gains elev_scalar; _csf_classify wires bSloopSmooth.
  • filter.py: streaming call site passes float(config.elev_scalar).
  • Iterate until the full suite is green; uphill-edge contingency as scoped in Key decisions. New tests must not be weakened — miscalibration gets reported, not edited.

Hard constraints: O(grid) streaming memory, no chunk-order dependence, streaming bit-identical to in-memory, CPU NumPy/SciPy only, frozen config, TableclothConfigError for validation. Rejected by research (do not implement): per-cell morphological slope estimation, global plane detrend, full PMF window schedules, trajectory-aligned anisotropic morphology.

Phase 3 — REVIEW: parallel panel Opus Sonnet Grok Sol
  • Opus — correctness: bit-exactness of the 0.0 path, gradient axis order/spacing, NaN and single-cell grids, bilinear clipping, parity, morphology-loop changes (if any) still remove crop-edge objects.
  • Sonnet — constraints checklist: JSON/dataclass parity, frozen config, coercion routing, O(grid) streaming, no new deps, style hygiene, streaming call site actually passes elev_scalar.
  • Grok — test adequacy: does each work item have a test that would fail on pre-change code; assertion strength; vacuous-pass guards (.any() on evaluation masks).
  • Sol — second opinion (fresh Codex session, no shared context with the implementer): re-derive the gate semantics vs Pingel/PDAL, hunt for real-segment regressions the synthetic tests can't see, judge the uphill-edge decision.

Findings classified must_fix / nice_to_have; review runs on the staged diff, read-only.

Phase 4 — FIX: apply must-fix findings Grok 4.5

Only runs if the panel produced must_fix findings. Applies them, keeps the suite green, may skip a wrong finding with justification.

Phase 5 — SHIP: verify + PR Fable
  • Independent full-suite run + my own read of the final diff (verify / code-review discipline).
  • Commit(s) in repo style AI3D-337: <imperative summary> (tests and implementation separated if the diff reads better that way).
  • Push branch, open draft PR. Never push main. No version-bump/Nexus publish unless asked (wrap-up exists for that).
Phase 6 — A/B on real data + visual QA done 2026-07-17

Ran on battlebox against source data: all 25 records of Abschnitt 4/5 segment_066 (lip acceptance) + segment_071 (vehicle removal), elev_scalar 0.0 vs 1.25, ~2.6 s and ~0.37 GB RSS per 1.67M-point record.

  • Kept-fraction delta: mean +3.7 pp (range +0.5 to +4.0 pp) across 24 of 25 records; outlier 071/Record003 +27.9 pp (0.448 → 0.727).
  • Record003 inspected (elevation profiles + top-down, kept/removed colored): a 5×2 m steep embankment face (~1.1 m rise) — the fixed gate kept only the bottom ~0.3 m; the scaled gate follows the face. Recovered points are genuine ground; still-removed points are the upper face and the record-edge dig band.
  • No object-removal regression seen: the median guardrail stripe in 066/Record023 stays removed under both configs; no vehicle blobs appeared in the 1.25 overlays.
  • Residual improvement opportunity: the record-boundary morphology dig band survives at both settings (thin red strips at strip edges). A surface-side fix — padding the seed grid at record boundaries or overlap-stitching adjacent records — is the natural follow-up ticket.
  • Default-on verdict: 1.25 looks safe-positive on these two segments, but gate the decision on downstream lane-finding acceptance (e.g. against cross_sections_run7_lanes truth) across more segments, not kept-fraction alone.
Phase 7 — precision QA + boundary dig-band fix done 2026-07-17
  • Precision (visual, 61 middle-segment records 064/068/070/073/076): zero confirmed object leakage at elev_scalar=1.25 — Opus-adjudicated across 19 flagged renders. Recovered mass is always shoulder/verge/V-ditch/embankment ground; cars, guardrail posts, signs, vegetation stay removed; zero points lost. Fixture cross-check: 100% object removal at 1.25 (guardrail, vegetation, vehicle flat + 18%).
  • smrf_edge_pad_enabled (default false = bit-identical, spy-tested): slope-extrapolating seed-grid padding (per-row/col Theil–Sen + two-pass trimmed-residual refit, slope clamp, distance decay) around the opening loop, cropped back before gradient. TDD + 4-model review; Sol's adversarial probe (boundary-parked object shielding) drove the trimmed-refit design and a boundary-parallel raised-strip regression test. Suite: 60 passed.
  • Re-validation (066+071, 1.25 vs 1.25+pad): dig-band record 071/Record003 goes 0.727 → 0.9835 (0.448 originally); all other records within ±0.16 pp, recovered points ~5–7 cm above ground, none >0.5 m.
  • Lane-truth acceptance gate: lane-point ground coverage (kept ground within 0.3 m XY / 0.2 m Z of each cross_sections_run7_lanes point) is bit-identical across 0.0 / 1.25 / 1.25+pad — all deltas are off-lane, so downstream lane-finding is unaffected. Recommendation: elev_scalar=1.25 + smrf_edge_pad_enabled=true are safe to enable together pending team sign-off; defaults stay off in this PR.

Current state

Worktree ai3d-337-slope-handling is clean (the earlier workflow launch was stopped before its first stage wrote anything). Deps synced; CSF extra confirmed absent locally. The stopped run (wf_4937f0ed-53e) can be resumed or relaunched on approval — nothing is cached that would go stale.

Execution environment

The local machine is thermally constrained, so all pytest runs execute on battlebox (24-core WSL over VPN): a wrapper script tar-pipes the worktree to /home/ai/ai3d-337-tablecloth, runs uv run pytest there, and streams results back. Edits stay local (single source of truth); only test execution is offloaded. Every workflow agent is instructed to use the wrapper and never run the suite locally. Fully moving the Codex/cursor delegates to battlebox was considered and rejected for this run — it would need two-way file sync with divergence risk.

Risks

Open questions all answered 2026-07-17