Handoff: pretrain on HelixNet for the Kirioll highway classifier
Continuation target
Produce a supervised-pretrained point-cloud semantic segmentation checkpoint on HelixNet (the only fully license-verified, commercially usable dense-label LiDAR dataset we found — CC-BY-4.0) that we will later fine-tune on 2–5 km of our own annotated highway MLS corridor. The checkpoint, its exact config, and its input-feature contract are the deliverables — not a usable highway classifier by itself.
Context (all you need; the full research lives elsewhere)
- Project "Kirioll": semantic segmentation of survey-grade mobile-laser-scanning (MLS) highway corridors, feeding 2D-plan digitization. Strategy: pretrain on public data → fine-tune on small own sample → active-learning loop.
- Commercial product → we must NOT train on CC-BY-NC datasets (SemanticKITTI, KITTI-360, Toronto-3D, Paris-Lille-3D, Waymo, Argoverse 2, nuScenes are all non-commercial or unverified). HelixNet is CC-BY-4.0, verified at the Zenodo record below — attribution required in product docs.
- Model strategy: Superpoint Transformer (SPT) is our production candidate (single-GPU friendly, MIT); PTv3 via Pointcept (MIT) is the accuracy benchmark — needs an Ampere-or-newer GPU for FlashAttention. Pick per available hardware; PTv3's outdoor pipelines match per-sweep automotive data better than SPT's large-tile design.
- Training-harness note: the target machine has internal packages with PyTorch Lightning training harnesses — reuse them. SPT is natively Hydra + Lightning (its
src/train.pyis a LightningModule/DataModule stack), so it plugs into a Lightning workflow as-is. Pointcept is NOT Lightning — it ships its own trainer; if you want PTv3 inside the existing Lightning harness, import the model as a plaintorch.nn.Module(from Pointcept orgithub.com/Pointcept/PointTransformerV3) and wrap it in a LightningModule, reusing Pointcept only for its data transforms (GridSample, crops) and CE+Lovász losses. If harness reuse matters more than squeezing out the last mIoU, the SPT path is the smoother choice. - Boss-facing overview of the whole plan: miro-plans.pages.dev/pages/highway-pointcloud-segmentation-20260710 (includes the full reference list).
Dataset: HelixNet
| Item | Value |
|---|---|
| Zenodo record (license source of truth) | zenodo.org/records/6519817 — rights: CC-BY-4.0 |
| Paper | Loiseau, Aubry, Landrieu, Online Segmentation of LiDAR Sequences: Dataset and Algorithm, ECCV 2022 — arXiv:2206.08194 |
| Official repo (dataloaders, class defs) | github.com/romainloiseau/HelixNet |
| Content | ~10 billion points, dense automotive LiDAR sequences with per-point semantic labels + sensor rotation/fiber/timestamp attributes |
| Size / classes | Check on the Zenodo record before downloading — reserve disk accordingly (order of 100s of GB); take the class list from the official repo, do not guess it |
Steps
- Environment: Linux + CUDA GPU. PTv3 path needs Ampere-or-newer (FlashAttention) + CUDA ≥ 11.6, PyTorch ≥ 1.12; SPT path runs on 16 GB GPUs (official 11 GB configs exist).
git clone https://github.com/Pointcept/Pointcept # PTv3 path (MIT) # or git clone https://github.com/drprojects/superpoint_transformer # SPT path (MIT) - Download HelixNet from the Zenodo record (use
zenodo_get 6519817or the per-file URLs). Verify checksums. - Inspect the native format with the official HelixNet repo's dataloaders before converting. Note which attributes exist (xyz, intensity, ring/fiber, time, label).
- Convert to the trainer's layout. Pointcept's
DefaultDatasetwants per-sample dirs of.npyarrays namedcoord,strength(intensity),segment(labels); setfeat_keys/in_channelsto match. Critical contract: use geometry + intensity only (no RGB) — our highway MLS has no RGB, and the fine-tune must reuse the same input channels or the first layer breaks. - Sanity run: overfit ~10 sweeps until near-zero train loss — proves the data pipeline before burning GPU-days.
- Train on a train/val split by sequence (not random per-sweep!). Starting hyperparameters (extrapolated from the frameworks' outdoor/DALES configs, not gospel): grid size 0.05 m, CE + Lovász loss, AdamW, OneCycle; batch to GPU memory. Log per-class IoU, not just mean.
- Record intensity normalization (e.g. clip + scale constants) — it is part of the checkpoint contract for later fine-tuning.
Expected results — calibrate before you start
| What | Expectation | Basis |
|---|---|---|
| Val mIoU on HelixNet itself | Should land in the same family as published outdoor per-sweep benchmarks (60s–70s mIoU); treat the paper's own baselines as the reference once you read them | No verified PTv3/SPT-on-HelixNet number exists — do not promise a target; measure and report |
| Value for the low-label fine-tune (the actual point) | Large gains in the small-label regime: published pretraining results show ~+14 mIoU at 1% labels (56.3 vs 42.2, CLIP2Scene on nuScenes) and 45.3-vs-26.0 (Sonata, indoor); at 100% labels the gain shrinks to ~1–2 points | arXiv:2301.04926, arXiv:2503.16429 |
| Convergence speed downstream | Fine-tuning from this checkpoint should converge in a fraction of the epochs of from-scratch training | Standard transfer-learning behavior reported across the pretraining literature above |
| Which classes transfer | Strong: road/ground, vegetation, poles, barriers, vehicles (geometry-dominant, present in HelixNet). Weak/none: Kirioll-specific rare classes (noise barriers, drains, shafts, call boxes) — those come from our own labels later, pretraining only provides better features | Class overlap analysis from the research phase |
| Honest unknown | HelixNet is per-sweep automotive LiDAR; our target is dense accumulated survey MLS (~10–50× denser). Domain-gap papers report the shift is real but bridgeable (adaptation techniques recover up to +20 mIoU). The transfer gain could shrink — the first Kirioll fine-tune is the measurement, and grid-sampling both domains to the same voxel size is the first mitigation | arXiv:2003.01174, arXiv:2407.06043 |
What this pretraining does NOT produce: a usable highway classifier. Out of the box the checkpoint segments HelixNet's classes on automotive-style sweeps. Its value is as a warm start that makes our 2–5 km of hand-labeled highway data go 5–20× further.
Deliverables & verification
- Best checkpoint + the exact training config + the dataset-conversion script + intensity-normalization constants (one archive).
- Training curves and final per-class IoU table on the held-out sequences.
- A short README stating: framework commit hash, input-feature contract (channels, voxel size), class list, and the CC-BY-4.0 attribution line for HelixNet.
- Verification bar: sanity-overfit passed; val mIoU stable (no divergence); per-class IoU reported for every class, including the bad ones.
Risks / open questions
- check Dataset size vs. disk/bandwidth on the target machine — verify on Zenodo before starting.
- check PTv3 without an Ampere+ GPU: set
enable_flash=falseand reduceenc/dec_patch_sizeto ~128 — slower; consider the SPT path instead. - decide Sweep-level vs. accumulated-scan training: accumulating consecutive sweeps densifies toward our MLS domain but departs from HelixNet's published setup. Default: train per published setup first; accumulate as a second experiment if time allows.
- note If self-supervised pretraining on our own unlabeled corridors is on the table later (Sonata-style — license-clean), this supervised HelixNet run is still useful as the comparison baseline.