Merge dashed lanes across segment boundaries

2026-09-01 · Step 7 (iolabs-point-cloud-modelling-lines) · generated by Claude

Summary

Dashed Lanes often end at segment_* boundaries even though a whole-dataset merge pass (merge_lane_fragments) exists and already fixed the same symptom for solid lines. The break happens at lane-identity level: stage 1 skeletonizes each segment independently, and stage 2's merge pass fails to chain the dashed tail/head pair at the boundary — but which gate rejects it (heading, lateral, overlap-duplicate, or type) is only logged at DEBUG level and no real run with the failure exists locally.

status 2026-09-03 Phase 1 done on Battlebox (Abschnitt 1, segments 0–21): no merge-gate failure reproduced — see Phase 1 results. Fix menu dropped. Phase 2 in progress (2026-09-03): boundary dash fusion — stage 1 flags boundary dashes and stores their clusters, stage 2 refits the two halves into one segment.

Plan: diagnose first, then fix the confirmed gate. Phase 1 builds a snapshot-driven boundary-merge analyzer and reruns stage 2 (seconds, no point clouds needed) on a real Abschnitt to name the failing gate per boundary. Phase 2 applies the matching fix from a prepared menu of four dash-specific candidates, all inside the shared merge pass. Phases 3–4: tests (incl. dashed case in the two-stage-equivalence harness), real-run validation, release.

Analysis — how dashed lanes break at boundaries

Current flow

All geometry lives in one global frame (geoshift is metadata only) — endpoints from different segment folders are directly comparable. No transform problem.
Two-stage runs (the production shape, per logs/s7a/s7b): stage 1 runs a fresh stepper per segment_* folder, so every lane ends at its segment boundary by construction. Stage 2 concatenates partial snapshots (load_partial_lane_state_snapshots, cluster_stepper.py:3588) and relies on the global reduce _connect_and_axis (:3901) to recover the joins.
The recovery pass is merge_lane_fragments (lane_linking.py:592): greedy best-score tail→head chaining under widened gates (merge_max_along_gap 80 m, merge_lateral_threshold 0.6 m + 0.02/m slack capped at 1.5 m, merge_heading_threshold_degrees 20°), plus a duplicate-overlap link path and duplicate-segment collapse. One predecessor/successor per lane, ≤5 fixpoint rounds.
This pass already works for solid lines — docs/2d-lane-connection.md documents the historical inverse: solid lines fragmented per segment, short dashes survived. After the 2D rework, the symptom moved to dashed lines.
A dedicated _reconnect_across_segment_boundaries pass exists but is off by default (two_stage.boundary_reconnect: false) and is redundant with the merge pass (same scorer, tighter gates).

Suspects (dash-specific, from code reading)

#	Suspect	Mechanism
S1	Overlap dead zone	A boundary-straddling dash skeletonized in both segments makes the head start behind the tail end. `continuation_match_2d` rejects any overlap > 1 m; the rescue path `_duplicate_overlap_link_score` (`lane_linking.py:165`) then demands 70% of the head's overlapping polyline samples lie within 0.7 m of the tail polyline — but dashed polylines are sparse and phase-shifted (samples sit on dashes, arc length includes gap jumps), so coverage can fail where a solid line trivially passes. Result: pair rejected by both paths → permanent break.
S2	End-direction noise on short fragments	Directions are fitted over 5 m of polyline arc (`lane_ends_2d`). A next-segment lane whose first dash is short/clipped yields a noisy head direction; merge heading gate (20°) is tighter than the online gate (25°).
S3	Type gate	`same_type_only` + `lane_dominant_type` (DASHED if any segment dashed): if Step 6 misclassifies the boundary fragment's `segment_type`, the merge is categorically blocked.
S4	Missing boundary dash	A dash clipped by the segment crop can fall under `min_cluster_size` (200 pts) in both folders and vanish, doubling the gap; combined with lateral drift on curves the lateral gate (0.6 m + slack) can miss.
S5	Greedy competition	One-successor-per-lane: a duplicate chain or parallel fragment can consume the tail's only successor slot before the true continuation is considered.

Diagnosis blocker: gate values for rejected near-misses exist (_log_merge_near_miss) but only at DEBUG, and local logs are a 3-segment toy run. Phase 1 removes this blindness.

Phase 1 results (2026-09-03) — no merge-gate failure found diagnosed

How a dashed lane crosses a segment boundary: stage 1 cuts lanes at the plane, stage 2 merge links tail to head through the gates, and the three cases that correctly stay unmerged — How it works in the end: stage 1 cuts every lane at the plane and detects the straddling dash twice; stage 2 links tail → head through the gates (type, gap, heading, lateral, overlap/duplicate trim) and now logs what fails; type flips, lane splits and junctions stay unmerged by design.

What joins the two half-dashes at a plane: stage 1 hands over two Lane objects with the boundary dash cut in half, one merge round collapses duplicates, gates tail to head and concatenates the segment lists, and the export writes the boundary dash as two touching line elements — What "merging the two dashes" does today: nothing geometric. The boundary dash is fitted as two halves (one per segment), the merge appends Lane B's segment list to Lane A's, and the export writes the two halves as two touching, collinear `<line>` elements under one feature id. Decided 2026-09-03: next phase fuses them into one segment.

Setup. Battlebox, Abschnitt 1 branch_000, run3 points on D:, Steps 4–6 rerun (CPU; CUDA segfaults in WSL, mask-clustering 0.2.4), Step 7 stage 1 per segment for segment_000..021, stage 2 gather on all 22 partials, plus the July production export (run7_lanes_branch_000_20260722, 131 segments, modelling-lines 0.12.1) re-analysed with scripts/find_boundary_breaks_xml.py. Merge gates: packaged defaults.

Evidence	Dashed tail/head pairs at boundaries	Result
Stage-1 partials 0–21, real `merge_lane_fragments` replayed (`analyze_boundary_merges.py --partials-dir`)	227 OK · 63 TYPE · 13 HEADING · 5 LATERAL · 4 OVERLAP+HEADING · 2 GAP · 1 OVERLAP-COVERAGE	Every dashed lane longer than one dash with a same-type continuation merged (clean 12 m gaps, 24–32 m missing-dash gaps, straddling dashes split in half at the plane, along = 0.0). All non-OK verdicts are different lines (lateral > 1.8 m), the crossing road at plane 16 (headings 80–180°), or 1–3 m stray fragments.
Stage-2 export 0–21 (`find_boundary_breaks_xml.py`)	38 features (20 dashed)	0 dashed breaks, 0 solid breaks; 2 dashed↔solid type flips near plane 20 (11 m apart, mid-segment).
July production export, all 131 segments	117 features (57 dashed)	0 dashed breaks; 6 type flips (planes 12, 20, 53, 55 — 17–25 m from the plane, i.e. Step 6 label changes, not boundary effects); dashed features ending with no continuation only at planes 16, 57, 125.
Abschnitt 4_5 July export	—	Same picture: no dashed↔dashed breaks; type flips at 43, 135; no-continuation ends at 18, 38, 44.

Plane 16 (the worst-looking case in the July export, three dashed separators ending at once) is a road junction. The surveyed trajectory turns ~70° between segment 15 and 16; segment 16 is an overpass with a crossing road in the tile. The dashed lanes that "end at the plane" belong to the road being left; the three that continue merge fine (headings 1–3°). Two solid lines ending at plane 15 have no detection at all in segment 15 (tile/corridor coverage), so nothing exists to merge.

Stage-1 lanes of segments 15-17 with boundary planes; the trajectory bends sharply at plane 16 — Stage-1 lanes (brown = segment 15, pink = 16, grey = 17), dashed planes = segment limits. The bend at "end seg 15" is plane 16.

Segment 16 intensity tile with Step 5 line mask in red — Segment 16 Step 4 tile with the Step 5 mask (red): overpass, crossing road bottom-left, guardrail picked up as one 271 m "dash".

Plane 20 (the only rejected multi-dash pair in the fresh run: lane 244 → 252, LATERAL 1.85 m vs cap 0.83 m) is a lane split. The 5-dash lane ends 11.5 m before the plane and keeps a flat heading while the carriageway and its dashed lanes bend away by ~10°; two solid lines end in the same 20 m and a new dashed separator begins there. The July export shows the same solid↔dashed handover at this plane. Merging would kink two different lines together, so the lateral gate is right to reject it.

Stage-1 lanes of segments 19 and 20 in plane-20 local coordinates — Segments 19–20 around plane 20 (x = along plane normal, y = lateral). Orange = DASHED, blue = SOLID, thick = dash. Lane 244 ends flat at x ≈ −12 while the road bends up-right. Made with `scripts/plot_boundary_lanes.py`.

Conclusion. With v0.12.1 the whole-dataset merge already chains dashed lanes across boundaries; suspects S1–S5 did not materialise on 21 real boundaries, so the Phase 2 fix menu (F1–F4) is not applied — changing working gates without a failing case would be a blind change. What remains real and boundary-adjacent: (a) dashed→solid/solid→dashed label flips from Step 6 that split a physical line into two features (blocked by same_type_only, kept by decision 3), (b) markings not detected in the next segment (Step 4 tile coverage / Step 5 misses), (c) junction topology. None of these is a Step 7 merge-gate issue.

Delivered instead of F1–F4: classify_merge_rejection + per-round INFO near-miss summary (step7_merge_near_miss_summary) so the next report can be attributed from production logs without DEBUG; regression tests pinning the dashed boundary cases a–f (clean gap, straddling duplicate dash, short head on a curve, parallel lanes stay apart, missing dash, dash split at the plane); the two analyzer scripts. Suite: 210 passed (was 194); all dashed boundary cases pass on the current gates, so they are guards, not xfails; the two-stage parity fixture now has a dashed separator with a dash cut in half at each of its two boundaries. If Miro can point at a concrete export/segment where a dashed line ends at a plane with a same-type continuation on the other side, the analyzer names the gate in seconds and the matching F-fix follows.

Key decisions

Diagnose before fixing. The failing gate is not determinable from code alone; stage 2 reruns from partial Lane State Snapshots take seconds and need no point clouds, so iteration is cheap. low risk
Fix inside the shared merge pass (merge_lane_fragments), not by enabling boundary_reconnect — one code path serves monolithic and two-stage runs, matching the ADR's shared-reduce principle. default
Dash-aware, not globally wider gates. Prefer dashed-specific config overrides and dash-geometry-aware scoring over loosening thresholds for everything, to avoid bridging real discontinuities (exits, lane drops). effort
Keep same_type_only; treat type misclassification as an upstream/classification fix if diagnosis shows S3.
New config keys go through the contract: cluster_stepper.default.json + strict _config.py validation + tests together (AGENTS.md rule).

Phases

Phase 1 — Diagnosis tooling + real-run repro ~½ day

New scripts/analyze_boundary_merges.py: loads a Lane State Snapshot (or stage-1 partials) + per-segment plane lists, and for each boundary plane reports every dashed tail/head pair within ±80 m: along-gap, lateral, heading, dominant types, overlap length, duplicate-coverage ratio — and which gate rejects it. Pure offline, reuses lane_ends_2d/continuation_match_2d/_duplicate_overlap_link_score so it reports the real gates, not a reimplementation.
New scripts/plot_boundary_lanes.py: plots the stage-1 lanes of segments k−1/k around plane k in plane-local coordinates, labelled with the same lane indices the analyzer reports, so a rejected pair can be checked by eye in one command.
Promote merge rejection visibility: structured INFO summary per merge run — counts of near-misses by failing gate (extends _log_merge_near_miss); keeps per-pair detail at DEBUG.
Fetch a real failing Abschnitt's stage-1 partial snapshots (Azure blob via azure-cli skill, or Miro-provided local run dir), run stage 2 (s7b) locally with merge DEBUG on.
Deliverable: table naming the failing suspect (S1–S5) per broken boundary, with numbers.

Phase 2 — Boundary dash fusion (decided 2026-09-03) ~1 day

Phase 1 showed the merge gates are fine; the remaining defect is that the boundary dash is exported as two touching line segments (see the second drawing above). The fix menu F1–F4 is dropped. Instead the two halves are refitted as one dash from their point clouds:

Stage 1 flags boundary dashes and stores their clusters. In _write_partial_lane_state_snapshot every DASHED segment whose start_point/edge_point lies within dash_fusion.boundary_tolerance (0.5 m) of the segment's first/last Step 3 plane gets boundary_end = start|end|both. Their cluster positions are written to a sidecar <partial stem>_boundary_clusters.npz next to the partial JSON (key l{lane}_s{segment}, float32 N×3); the JSON records the file and key. Snapshot schema v4, v3 stays readable.
Stage 2 fuses touching dashes (new dash_fusion.py, hooked in _connect_and_axis right after _merge_lane_fragments, so the monolithic run gets it too). Adjacent DASHED pairs in one lane with gap ≤ 0.5 m, lateral ≤ 0.25 m, heading ≤ 5° are replaced by one segment: refit mode when both halves carry points (union cloud → the normal dashed fit + find_ends + find_width), axis mode otherwise (one DashedLineAxis from the first start to the last end, width from the better-supported half). Real dash gaps are ≥ 3 m, so the gates cannot join two genuine dashes.
Wiring. The orchestrator (3dai.iolabs.orchestrator, s7a_skeletonize/s7b_connect_axis) passes only the partial path and the partials folder, so the sidecar convention needs no orchestrator code; only the s7_cluster_stepper env pin moves to the new release (branch ai3d-318-dash-fusion-pin). 7B stays on CPU: a refit is a few thousand points per plane.
New config section dash_fusion (enabled, max_gap, max_lateral, max_heading_degrees, boundary_tolerance, store_boundary_clusters) in cluster_stepper.default.json + _config_model.py.
Logging: step7_partial_boundary_clusters (stage 1: flagged segments, points) and step7_dash_fusion_summary (fused / refit / axis counts).

implemented 2026-09-03 Branch ai3d-318-merge-dashed-boundaries (rebased on v0.12.2), PR #11. Two adversarial reviews (Opus, Codex) plus a final pass found and fixed: a crash of the monolithic lane-points export on fused segments; the seam ends were read from start_point/edge_point, which LaneSegment.reverse() never swapped, so reversed halves silently never fused — ends now come from the axis and reverse() turns every end-anchored field; the boundary flags are now part of the gate and are recomputed in _connect_and_axis for both run modes, so monolithic and two-stage cannot diverge; refit exceptions fall back to axis mode; stale or corrupt sidecars are tolerated. Suite: 264 tests.

Side effect worth knowing. The old reverse() returned an unnormalised direction (~1e-4 long), so Lane.process_cluster measured a ~90° turn and rejected every backwards-fitted cluster it had just reversed (MAX_DIRECTION_CHANGE_EXCEEDED). With the fix those clusters are kept. This changes online lane building beyond the fusion itself and is validated on Battlebox below (lane counts before/after).

Real data, axis mode (old v3 partials, no sidecars): Abschnitt 1 segments 0–21, stage 2 fused 19 pairs in 9 lanes; the XML went from 828 to 809 <line> elements in the same 41 features, i.e. one line per fused pair. Refit-mode numbers from the rerun with sidecars: see Phase 4 (19 pairs, all refit, same 808 lines).

Phase 3 — Tests ~½ day

Done in Phase 1: dashed boundary cases a–f in tests/test_lane_linking.py.
New tests/test_dash_fusion.py: axis-mode and refit-mode fusion, three-piece chain, rejections (3 m gap, 0.5 m lateral, 15°, DASHED next to SOLID), disabled config.
tests/test_lane_state.py: boundary flag + sidecar round trip, v3 snapshot still readable, missing sidecar degrades to axis mode.
tests/test_two_stage_split.py parity: the dashed separator (dash cut in half at planes x=10 and x=20) must come out with 4 segments in both modes, one segment spanning each plane, none ending on it; stage-1 CLI test checks the sidecar.
Config tests for new keys (defaults + strict-validation rejection), per _config.py contract.
Snapshot/label tests only if merge order perturbs canonical numbering (test_lane_labels.py).

Phase 4 — Validation, docs, release ~½ day

done 2026-09-03 Battlebox, Abschnitt 1 segments 0–21, stage 1 rerun with sidecars (95–165 s per segment on CPU, 21 sidecars, 2.3 MB, 72 flagged dashes) + stage 2:

Run	Stage-1 lanes / segments	Fused pairs	Mode	XML `<line>` elements
v0.12.1 partials, old stage 2	266 / 841	0	—	827
v0.12.1 partials, new stage 2	266 / 841	19 in 9 lanes	axis	808
rerun stage 1 + new stage 2	266 / 841	19 in 9 lanes	refit	808

All 19 refitted dashes span their plane, are 5.95–7.4 m long and carry a fitted width (0.15 m, 0.30 m for the wide separators). No flagged facing pair was left unfused; the other flagged dashes simply end near a plane with a normal gap beyond it. Stage-1 lane and segment counts are identical before and after the reverse() fix, so that side effect did not change lane building on this branch.

Orchestrator: bump environments/s7_cluster_stepper pin to the new modelling-lines release and the env version, fix the "no point clouds in partial snapshots" comments (branch ai3d-318-dash-fusion-pin, worktree 3dai.iolabs.orchestrator-ai3d-318).
Verify no regressions: solid-line merges unchanged, no cross-lane false merges (lateral profile check).
Update docs/2d-lane-connection.md with the dashed-boundary chapter (it already narrates the solid-line fix).
Release via /wrap-up (version bump, Nexus publish, Jira-tagged commit) once validated.

Risks

false merges Wider dash gates can bridge real discontinuities (exit gores, lane drops, dash→solid transitions). Mitigation: dash-period-bounded gaps, keep max_lateral cap, parallel-line rejection tests, real-run lateral-profile check.
label churn Merging changes lane counts → canonical Lane Labels and export feature IDs shift. Downstream consumers of XML IDs must not assume stability across versions (they already can't, but note it in the release).
monolithic latch In monolithic runs the >50 m NOT_CONTINUED latch (plane_distance_from_lane_threshold) can kill dashed lanes over long paint gaps independent of boundaries; the merge pass is also the safety net there — Phase 1 will show if it contributes.
data access No local failing run; Phase 1 depends on pulling a real Abschnitt's snapshots (Azure creds / blob availability).
Step 6 upstream If diagnosis shows missing boundary dashes (S4 via min_cluster_size) or type misclassification (S3), the clean fix is partly upstream (Step 6 clustering/typing) — scope decision needed then.

Open questions

Which run should Phase 1 diagnose against?
default Pull the latest failing Abschnitt's stage-1 partial snapshots from the Azure blob outputs (tell me which Abschnitt shows it worst).

alt You point me at a local run directory with run7 outputs.
Where do you observe the breaks — so I validate at the right level?
default Exported XML / top-down overlay: dashed feature ends at the boundary (lane-identity problem, as planned).

alt Only in debug lane labels / snapshot — then it may be cosmetic and the export is already merged.
May dashed↔solid merge at a boundary when geometry is perfect (S3 misclassification case)?
default No — keep same_type_only; fix classification instead if S3 shows up.

alt Allow cross-type only within ±20 m of a boundary plane, exported type by majority length.
What to do with the redundant, disabled two_stage.boundary_reconnect pass?
default Leave untouched in this change (out of scope); revisit removal after the merge pass handles boundaries.

alt Delete it and its config key now as part of Phase 2 cleanup.