Round-1 report — Azure retrain on 260624_confirmed_good

2026-07-11 · AI3D-338 · linebitmapsegmentation · program plan

Summary

Both round-1 anchors reproduce their Phase-D scores almost exactly on the 2× bigger dataset — the ~0.87 pixel-F1 ceiling did not move. dice_focal 0.8750 (was 0.8712), focal_tversky+r34 0.8678 (was 0.8683), now measured on a 97-tile test set vs the old 50. Doubling clean data neither lifted nor dented headline F1, while calibration and topology stayed excellent (clDice 0.97–0.98, dice_focal area ratios 1.03/1.00). This is strong evidence for Phase D's #1 suspect: the fixed 4 px label stroke, not data volume and not the model, is what caps pixel-F1. Round 2 attacks exactly that (per-class stroke at physical paint width — the rasterizer only paints odd widths, so the realizable correction is solid 5 / dashed 3 px), plus a longer schedule for the r34 run that hit its 60-epoch cap without early-stopping.

Results (held-out 97-tile test)

run	loss / encoder	F1 fg	F1 S/D	clDice	cov S/D	area S/D	best ep
`dice_focal_260624`	dice_focal / r18	0.8750	0.863 / 0.887	0.971	0.875 / 0.889	1.03 / 1.00	35 / 46 early stop
`ft_r34_260624`	focal_tversky / r34	0.8678	0.853 / 0.882	0.975	0.916 / 0.938	1.15 / 1.13	54 / 60 hit cap
Phase-D refs (old 50-tile test)	same configs	0.8712 / 0.8683	—	~0.96 / 0.975	— / 0.93·0.94	0.98·1.03 / 1.19·1.12	—

Overlay review (best-F1 checkpoints, 6-tile sheets)

Predictions track labels closely on every sampled tile for both runs: continuous solids through occlusion shadows, discrete well-placed dashes, no noise hallucinations.
Neither Phase-D structural failure appeared in the samples — no asphalt-edge false-positive line, no gore over-paint (6/97 tiles caveat).
The known loss personalities are intact on new data: ft_r34 recovers the faint dashes on the dark adjacent-carriageway band that dice_focal drops (recall edge), but paints ~15% fat; dice_focal is near-perfectly area-calibrated but leaves faint paint behind.

What it means

Data volume was not the pixel-F1 lever. +96% tiles moved headline F1 by +0.004 / −0.0005 — deep inside the ±0.035 seed band. The Phase-D "data is the bottleneck" verdict referred to label-geometry and coverage of hard cases; simply adding more of the same tiles confirms the ceiling is structural, not statistical.
The 4 px label stroke is now the prime suspect for the ~0.87 cap: measured paint is ~5 px solid / ~3.5–4 px dashed, the two loss families disagree with the label in opposite directions (under- vs over-paint), and both hold exactly at the same F1 on a completely different test set. A label the model cannot match caps F1 regardless of data volume — untested until round 2.
Topology is solved and stays solved (clDice 0.97–0.98 on 2× data). For the width-invariant CAD centerline goal, ft_r34_260624 is already a shippable front-end: best coverage (0.916/0.938) with harmless over-paint.
Generalisation improved silently: the same scores now hold on a test set with new sections (Abschnitt_4_5, more of 2/3) — the model covers a wider road inventory at unchanged quality.
ft_r34's best epoch was 54/60 with val still creeping — the 60-epoch cap binds on 2× data; the schedule needs headroom before its result is trusted.

Round 2 — design

#	run	change vs round 1	question it answers
1	`dice_focal_260624_s5d3`	per-class label stroke: solid 5 / dashed 3 px	Is the 4 px label the F1 ceiling? If F1 jumps on either anchor → yes, keep per-class stroke and re-tune. If flat on both → label geometry beyond stroke width (gores, faint paint) is the bottleneck → data verdict, program exit criterion approaches.
2	`ft_r34_260624_s5d3`	per-class stroke + max_epochs 90
3	`ft_r34_260624_e90`	max_epochs 90 only (4 px stroke)	Was the 60-epoch cap costing the r34 run real F1? Also isolates schedule from stroke in run 2.

The per-class stroke needs a small harness change (label_stroke_px accepting {solid: 5, dashed: 3} through config → tiles → rasterize → evaluate) — implementation delegated to Grok 4.5 in Cursor per Miro's instruction, reviewed here before submission. Training labels change, so metrics of stroke runs are compared on trend and coverage/tightness, and additionally re-scored against the 4 px test labels for a fixed yardstick.

held for round 3 seed repeats (noise floor on new data), Tversky α dial, r50/unet++ capacity probe — only if round 2 leaves headline F1 ambiguous.

Implementation note — stroke quantization

While wiring the per-class stroke, Grok 4.5's verification surfaced a fact that reframes the stroke hypothesis: _cv_thickness quantizes every requested width to an odd painted width (cv2 cannot center an even stroke), and 4 rounds up — so the round-1 "4 px" labels already painted 5 px. Solid labels have therefore matched the measured physical paint (~5 px) all along; it is the dashed labels that are fat (painted 5 px vs measured ~3.5 px). The realizable correction is {solid: 5, dashed: 3} — solid unchanged, dashed thinned — which is what round 2 runs. This weakens the "label too thin for solids" reading of the sweep2 report and sharpens the alternative: if F1 stays flat in round 2, the residual gap is label geometry beyond stroke width (gores, faint paint, ambiguous edges) — the program's data-bottleneck exit.

Artifacts

Per-experiment markdown reports (metrics tables, control curves, overlays, visual assessment): docs/experiments/round1_260624/<run>/report.md on branch worktree-azure-retrain-260624.
AML jobs: linebitmapseg_ft_r34_260624_20260711t050704z, linebitmapseg_dice_focal_260624_20260711t051016z (workspace ai3d-lf-mlw-pc-01); TensorBoard servable via scripts/azure/tb_azure.sh <job>[,<job>].
Best checkpoints downloaded to runs/<exp>/version_0/checkpoints/ (main checkout); DVC promotion pending the round-2 outcome (promote the program's final pick, not every round).