Agent handoff 3 / 4 self-contained — no vault access needed

Implement tree & greenery detection — new repo

2026-07-10 · for a fresh implementation agent on a machine with repo + Nexus access, without the knowledge vault · all research findings inlined · the one class of four where (light) ML is justified

Continuation target: new repo implementing vegetation detection in highway MLS segments: ground normalization → multi-scale covariance ("eigen") features → Random Forest classifier → tree/shrub instances with positions & extents, exported as XML. Deep learning (PointMetaBase) is the fallback, not v1. PDAL provides every classic filter needed — do not reimplement them.

Project context (inlined)

Pipeline: LAS highway scans → trajectory → segments cut along the road axis (numpy .npz; XYZ + intensity + RGB) → per-segment detectors → XML for CAD.
Your input: segment clouds + division planes (iolabs-point-cloud-segmentation-trajectory), road-axis spline, per-segment road-surface points/meshes (iolabs-point-cloud-filtering-surface) — the surface covers the road; roadside terrain needs its own ground filter (CSF, below).
Data prerequisite: ask Miro to ship 3–5 sample "Abschnitt 1" segments incl. trajectory + surface outputs.
Neighbor: a vertical-signs agent handles poles/signs. The tree-vs-pole boundary is shared — agree on an "ambiguous cluster" exchange format (cluster id, features, both candidate labels).

Research findings (inlined — adversarially fact-checked 2026-07-07)

Confirmed:

Handcrafted features + Random Forest are competitive (2-1 vote on the "no DL needed" reading — treat as strong baseline, not ceiling): 18 eigen/height features + RF reach 95.36% overall accuracy (kappa 0.89) on the Oakland benchmark incl. Vegetation and Pole/Trunk classes [Weinmann 2017]. The most discriminative features across studies: omnivariance, eigenentropy, change of curvature, max height difference, std of heights, 2D eigenvalue ratio — that's your feature shortlist.
Multi-scale matters: computing the line/plane/volume character of each point's neighborhood at several radii separates vegetation at >98% in natural terrestrial scenes [Brodu & Lague 2011] — this is the CANUPO method, available as a CloudCompare plugin (useful to bootstrap labels manually).
If DL becomes necessary: benchmark of 7 architectures on 3 MLS datasets (78–222M points) for binary vegetation segmentation: PointMetaBase 95.24% mIoU (Chandigarh), KPConv omni-supervised 91.26% (Toronto3D), PointCNN 85.68% (Kerala) — no single model wins across datasets; the authors recommend PointMetaBase (2.7M params, vs SPoTr's 66.4M) for limited compute [Aditya 2023]. Note: benchmark ran on a 32 GB V100; "fits a 16 GB card" is an inference, verify.
Ground filtering: CSF (cloth-simulation filter) is the standard classic ground/non-ground split; available as PDAL filters.csf and standalone (jianboqi/CSF, pip-installable). Height-above-ground via PDAL filters.hag_delaunay.

Refuted claims — do NOT use:

A single density threshold (~15 pts/m²) separating poles from trees — rejected 0-3.
Eigenvalue isotropy alone splitting trees from signs — rejected 1-2.

Every single-feature vegetation separator died in fact-checking. The multi-feature bundle is the method.

Open: label budget is the real constraint — RF needs little, start tiny and grow only if metrics demand; urban benchmarks → Autobahn transfer is unproven; seasonal leaf state of our scans vs training patches; ask Miro what the CAD deliverable for greenery is (tree points? crown circles? hedge polygons?) — it drives the export schema.

Artifacts to reuse

Nexus index: https://nexus.iolabs.ch/repository/pypi-private/simple (copy [tool.uv.sources] from 3dai.lanefinder). Flat naming: iolabs-point-cloud-* → import iolabs_point_cloud_*. Update: uv lock --upgrade-package … && uv sync.

Artifact	Use for
`iolabs-point-cloud-segmentation-trajectory` (Nexus)	Segment clouds (input)
`iolabs-point-cloud-filtering-surface` (Nexus)	Road surface; height-above-road inside the corridor
PDAL: `filters.csf`, `filters.hag_delaunay`, `filters.covariancefeatures`, `filters.dbscan`	Ground, HAG, eigen-features at multiple radii, clustering
`iolabs-point-cloud-filtering-clusters` (Nexus)	Cluster post-processing / density filters
scikit-learn RandomForest + joblib	The classifier; version the training set in-repo
`iolabs-point-cloud-modelling-export` · `iolabs-geometry-visualization` · `io_orchestrator.ray_step`	XML out, QC overlays, pipeline step

Suggested repo/package: 3dai.iolabs.pointcloud.vegetation / iolabs-point-cloud-detection-vegetation — confirm remote/org with Miro.

Algorithm to implement

Ground & height: CSF on the roadside strip → height-above-ground; keep HAG > ~0.3 m within a lateral corridor wider than the guardrail band (e.g. 0–20 m from axis).
Features: covariance features at 2–3 radii (e.g. 0.25 / 0.5 / 1.0 m): the six shortlisted features + verticality, scattering, planarity. PDAL covariancefeatures emits these directly.
Classify: RF on the bundle → vegetation / not-vegetation per point (or per small voxel for speed). Bootstrap labels with CANUPO in CloudCompare on the sample segments.
Instances: DBSCAN on vegetation points → clusters; trees vs shrubs by vertical extent + trunk/canopy profile (thin vertical base + volume above = tree); merge overlapping canopies conservatively.
Export: per-instance XML: axis station + offset, footprint extent, height, class, confidence (schema pending Miro's answer on the deliverable).
Fallback only if RF underperforms: PointMetaBase binary vegetation segmentation, benchmarked on our data first.

Verification

Hand-labelled vegetation masks on the sample segments; report point-level IoU + instance-level P/R. Bars: ~95% OA classic, 85–95% mIoU DL.
Confusion focus: trees vs poles/signs — exercise the shared ambiguous-cluster interface with the signs agent.

References (inlined)

Weinmann, Jutzi, Mallet & Weinmann (2017). Geometric Features and their Relevance for 3D Point Cloud Classification. ISPRS Annals IV-1/W1. DOI 10.5194/isprs-annals-IV-1-W1-157-2017
Brodu & Lague (2011). 3D terrestrial lidar classification using a multi-scale dimensionality criterion (CANUPO). arXiv:1107.0550
Aditya, Lohani, Aryal & Winter (2023). Benchmarking DL for Urban Vegetation MLS Segmentation. arXiv:2306.10274
CSF: Zhang et al. cloth-simulation filter — github.com/jianboqi/CSF; PDAL filters.csf, filters.covariancefeatures — pdal.io

Public research briefs: greenery & posts brief · all 30 sources.

Conventions

UV project, Python ≥3.11, lint/test setup copied from an existing 3dai.iolabs.* repo.
Release = version bump → uv build → publish to Nexus pypi-private → commit tagged with the Jira issue (project AI3D).
No vault on this machine: report findings/decisions back to Miro in your final summary.