Handoff: distributed CUDA compute for training and inference

Continuation target

Current state and settled facts

• Darkbloom is a public-alpha network for Apple Silicon providers, using an attested/operator-blind inference design. It is not an RTX 3080 Ti/CUDA provider option. Its official provider requirements: M1–M4, macOS 14+, 16 GB+ RAM recommended: Darkbloom provider setup.
• The local machine reported NVIDIA GeForce RTX 3080 Ti Laptop GPU, 16 GiB VRAM, CUDA 13.2. Current GPU power cap at observation: 45 W; default 125 W; maximum 150 W. This is a local observation, not a performance benchmark.
• Previous high-level conclusion: Salad is the low-friction experiment; Vast.ai is closest to renting CUDA capacity but needs a dedicated Ubuntu/networking posture; Runpod no longer accepts new Community Cloud hosts. Do not repeat this as established fact without rechecking current official documentation.
• Research was captured in /home/miro/zettelkasten/Personal notes/2026-08-28 CUDA GPU renting, mining, and winter heating.md. It is useful context but not a substitute for fresh verification.
• No platform account, workload, benchmark, deployment, or network configuration has been created. Do not create any during review.

Required review scope

Platforms to evaluate

Workloads to assess separately

• Small-model inference: intermittent, latency-sensitive or batch inference; VRAM/model-size limits; whether HTTP/serverless is available; cold starts; concurrency; model/image caching.
• Training: single-GPU fine-tuning versus multi-GPU/distributed training; checkpointing; interruption/preemption; persistent disks; reproducibility; expected need for 24+ GiB VRAM versus the local 16 GiB GPU.
• Home-hosted supply: feasibility, never as a recommended production data path. Address dedicated host, isolation, ISP policy/CGNAT/ports, uptime, and the risks of putting customer workloads on a personal machine.

Data movement is a hard requirement

For every serious candidate, answer these questions with current official documentation. Do not use vague claims such as “fast storage access.”

1. Which region/location does the compute run in, and can the user choose it? Can it be co-located with an Azure Storage account or S3 bucket?
2. What public ingress/egress billing applies? Separate provider-to-provider internet transfer, same-cloud/same-region transfer, and marketplace/network transfer. Quote the unit and the effective date.
3. How does a workload read Azure Blob and S3: HTTPS/SDK, signed URL/SAS, managed identity/workload identity, service principal/access keys, mounted filesystem (BlobFuse/S3 mount), or a platform integration? Which credential types are recommended and which are dangerous on an untrusted marketplace node?
4. What throughput is documented, guaranteed, or merely empirical? Distinguish WAN bandwidth, storage-account request limits, disk/cache throughput, and first-download versus cache hit. If no SLA/number is published, say so.
5. Can large data be staged onto persistent storage, cached between jobs, or copied once? What persists after termination? What volume/dataset transfer cost results?
6. For a concrete scale ladder—10 GB, 100 GB, 1 TB—estimate transfer time only when justified by documented bandwidth. Otherwise give the formula and a measurement plan. Include compression, sharding, and streaming options where relevant.
7. What data should not be sent to an untrusted/consumer node? Address client confidentiality, regulated data, model weights, access-token blast radius, encrypted-at-rest versus data-in-use, and deletion/retention controls.

Evidence standard and starting sources

Required Use first-party documentation for platform capability, requirements, storage authentication, pricing, network policies, and security claims. Capture access date and direct URL per meaningful claim. Label marketplace/vendor marketing and third-party benchmark evidence clearly. Current facts must be live-verified.

• Vast host setup: docs.vast.ai/host/hosting-overview; host FAQ: console.vast.ai/faq.
• Salad earnings/workload types: Salad support; security model: Salad container FAQ.
• TensorDock host/cloud documentation: docs.tensordock.com.
• Runpod current host status: Choose a Pod.
• Azure research must use Microsoft Learn/Azure pricing pages; AWS research must use AWS docs/pricing pages. Do not rely on blog posts for bills, quotas, or storage identity.

Do not log in, create accounts, consume paid cloud resources, download proprietary data, alter the vault, or make any deployment during review. The scope is read-only web/documentation research and a decision artifact.

Expected deliverable

1. Executive recommendation: a shortlist for small inference and another for training; specify when Azure wins outright.
2. A comparison table with: hardware/access model, GPU/VRAM availability, region choice, startup/availability, persistent storage, Azure Blob/S3 access method, ingress/egress costs, measured-or-documented bandwidth, security/trust model, operational burden, and pricing/payout caveats.
3. A data-path diagram for each recommended architecture, e.g. Azure Blob → same-region Azure GPU → checkpoint Blob versus Azure Blob → public internet → marketplace container → public internet → Blob.
4. A 10 GB / 100 GB / 1 TB data movement analysis, including where an estimate is an inference rather than a provider guarantee.
5. A minimal, reversible benchmark plan: one public dataset, a read-only bucket/container, bandwidth + time-to-first-batch + cache-hit measurements, total transfer accounting, and teardown steps. No execution unless separately approved.
6. A source list with direct URLs and access dates; a clear “unknown/not documented” row wherever applicable.

Suggested skills

research for primary-source investigation; azure-cli only if later authorized to inspect existing Azure ML/storage state; zettelkasten-vault only if asked to save the finished review to the vault. Browser research is mandatory for pricing, quotas, platform requirements, and current availability.