Research handoff: separating individual tree canopies

Continuation target

The desired output is a per-point tree_instance_id for canopy points, with explicit abstention when the evidence cannot support a reliable split. The goal is individual trees, not species classification.

Current state and settled facts

• The production context is roadside, side-looking MLS, not top-down airborne forestry. Expect one-sided occlusion, range-dependent density, trimmed crowns, hedges, conifers, poles, signs, fences, and merged vegetation blobs.
• The current 3D harness is semantic-first: eight roadside classes plus trained background; train ID 9 is ignored void. E10–E12 cover pseudo-labels and weak supervision but remain gated-later.
• The harness preserves stable source identity through origin_index and already carries obj_id in framework emissions. Research should define how an individual-tree artifact uses these identities without changing semantic ontology.
• The current classical tree path can miss conifers because its training labels encode broadleaf-like crown assumptions. Any bootstrap labels must deliberately include conifers and small isolated trees.
• A closely matching EPFL ESO project/paper used human ratings Single, Multi, and Non-tree to improve weakly supervised airborne tree-instance segmentation. It is relevant to candidate quality and pseudo-labeling, but its model, thresholds, and headline gain do not transfer directly to MLS.
• No public EPFL annotation dataset, extension code, or checkpoint was verified as of 2026-08-14. Asking the authors for the four fully annotated tiles, 13,316 cluster ratings, cluster masks, split manifests, and pseudo-label snapshots is a useful parallel action; do not send correspondence without user authorization.

Key local artifacts

• /home/miro/zettelkasten/Project notes/Point cloud analysis on linear structures/2026-08-14 EPFL weakly supervised tree segmentation - project status and greenery transfer.md
• /home/miro/zettelkasten/Project notes/Point cloud analysis on linear structures/2026-07-17 Conifer detection failure in the verticalsigns tree path.md
• /home/miro/zettelkasten/Project notes/Point cloud analysis on linear structures/2026-08-12 3D segmentation deep research - verified model landscape, experiment plan and prior-claim re-audit.md
• /home/miro/zettelkasten/Project notes/Point cloud analysis on linear structures/3D segmentation training harness.md
• /home/miro/dev/3dai.iolabs.pointcloud.mlsegmentation/docs/plans/2026-08-12-harness-implementation-spec.md
• /home/miro/dev/3dai.iolabs.pointcloud.mlsegmentation/docs/experiments/e10-pseudo-label-engine.md and e11-gold-plus-pseudo.md

Primary external anchors

• EPFL weakly supervised tree-instance paper
• EPFL project brief
• SegmentAnyTree official repository

Questions the next agent must answer

Research protocol

1. Use the vault workflow: search the named notes before writing and link the final note into the 3D & Point Cloud Analysis MOC.
2. Use primary sources only for substantive claims: official papers, repositories, licences, dataset pages, documentation, and release histories.
3. Prefer ground-based or mobile-laser evidence. Label airborne-only results as mechanism-level evidence; do not transfer accuracy numbers.
4. Trace public implementations end to end: required preprocessing, semantic assumptions, instance outputs, data format, model licence, weight licence, dataset licence, GPU/runtime, and whether inference accepts arbitrary LAS/LAZ.
5. Look for failure analyses, not just leaderboard values: touching crowns, suppressed understory, occluded stems, multi-stem trees, small conifers, scan-density shifts, and edge/tile artifacts.
6. Design a small gold set that can falsify the recommendation. Estimate the minimum useful number of tree groups and annotation form; justify any number rather than copying EPFL's scale.
7. Draft—but do not send—a short data request to the EPFL authors if their labels would materially answer a research question.
8. Write one source-backed vault report with a flat decision: first baseline, challenger, parked options, experiment sequence, go/stop rules, and unresolved blockers.

Minimum evaluation contract

• One-to-one instance matching with a predeclared criterion; report precision, recall, F1, matched IoU, and coverage.
• Separate counts/rates for under-segmentation (multiple gold trees merged), over-segmentation (one gold tree split), missed trees, and spurious tree instances.
• Evaluate on raw-source points after inverse mapping, not only voxels or cropped model inputs.
• Use corridor/campaign-held-out groups; repeated scans of the same trees must not cross train and evaluation.
• Report runtime, memory, points per second, and manual annotation minutes alongside quality.

Expected deliverable

Create:

/home/miro/zettelkasten/Project notes/Point cloud analysis on linear structures/
YYYY-MM-DD Individual tree canopy separation for roadside MLS - methods and experiment plan.md

The report must include:

• an executive decision with calibrated confidence;
• a comparison table of method families and verified implementations;
• explicit ALS→MLS transfer boundaries;
• the proposed instance artifact, for example:

origin_index: uint64[N]       # stable source identity
semantic_train_label: uint8[N]
tree_instance_id: int32[N]   # -1 outside/unknown, local IDs otherwise
instance_confidence: float32[K]
split_quality: enum[K]       # single, merged, partial, hedge, non-tree, uncertain
label_source: enum[K]        # gold, classical, model, reviewed-pseudo
producer_version: string
source_manifest_hash: string

• a smallest-useful gold annotation protocol;
• baseline/challenger experiment cards with exact metrics and gates;
• a recommended integration point relative to semantic vegetation detection and E10/E11;
• source URLs next to every material claim and a separate licence table.

Research how to split touching or overlapping roadside vegetation clusters into individual tree canopy instances in side-looking mobile laser-scanning point clouds. Compare geometric, learned, and hybrid methods using primary sources and current official implementations. Prioritize MLS/TLS evidence and explicitly bound all ALS transfer. Read the handoff's local artifacts first. Return one source-backed vault report with a first baseline, one challenger, artifact schema compatible with origin_index, a gold annotation protocol, matched-instance metrics, and go/stop rules. Do not implement code and do not send external messages.

Risks and open questions

• High A canopy may not be separable from one-sided MLS geometry when stems and crown valleys are occluded. The system needs an abstention state.
• High Current candidate clusters may omit conifers or merge tree-plus-hedge geometry, capping any downstream splitter.
• Medium “Individual tree” is ambiguous for multi-stem trees and continuous hedges; the gold ontology must settle this before annotation.
• Medium Forest-specific pretrained models may carry non-commercial weight/data restrictions even when code is permissively licensed.
• Medium Random tiles or repeated passes can leak the same tree geometry across splits.
• Opportunity The EPFL Single/Multi/Non-tree rating pattern could cheaply identify which current clusters need splitting before expensive point-wise annotation.

Suggested skills

• research for primary-source investigation and a repository-backed report.
• zettelkasten-vault for note search, writing, citations, and MOC integration.
• get-api-docs only if implementation planning requires current third-party framework APIs.
• codebase-design later, if the user asks to design the instance artifact/module boundary.