Project 02 / Decentralised GPU cloud
Decentralised
GPU services.
For a startup.
The project
Compute, distributed.
Access, connected.
- Client
- A startup
- Our work
- Decentralised GPU as a service
- Environment
- Decentralised: GPU servers run by 140+ independent providers
- Technology
- Kubernetes · Provider agents · Metering
- Across the stack
- Infrastructure
The service model
Connecting capacity
with workloads.
The service model connects distributed GPU resources with the workloads that need them.
Engineering spotlight / Decentralised compute
Many providers.
One service.
A platform that onboards independent GPU providers, verifies their hardware and schedules customer workloads across them, so customers see one service rather than many suppliers.
Independent providers
Onboarded through one agent and one verification path, in 11 countries.
GPUs listed
From RTX 4090 workstations to H100 servers, each benchmarked before listing.
From install to listed capacity
Agent install, hardware checks and reference benchmarks, fully automated.
Machines held back
At verification, before any customer workload reached them.
To fence a failed provider
Missed heartbeats stop new placements. Interrupted jobs are requeued.
Jobs metered
One usage record per job serves customer billing and provider payouts.
Client and provider details are anonymised.
The challenge
Capacity you do not own.
A service you stand behind.
Supply comes from many providers, each with its own hardware, network and reliability.
Mixed hardware.
Different GPU models, drivers and networks behind every provider.
Unverified supply.
Advertised capacity has to be checked before customers rely on it.
Two sides to every job.
Usage has to be metered for the customer who ran it and the provider who served it.
The operating stack
Every layer,
working together.
Kubernetes
NVIDIA
DockerA central control plane, and a lightweight agent on every provider machine. Observability and security run across both.
Application
Self-service API and console · Projects · API keys
Scheduling and metering
Capability-aware placement · Queues · Usage records
Control plane
Kubernetes · Postgres · Redis · Vault
Provider agent
Onboarding · Hardware verification · Heartbeats · Job runtime
Provider hardware
Heterogeneous NVIDIA GPUs · Container runtime
The network
Many providers.
Three tiers of capacity.
2,400+ GPUs from 140+ independent providers, admitted to a tier by benchmark rather than by claim.
H100 and A100.
Servers in professional facilities, for training and large-model inference.
L40S and RTX 6000 Ada.
Workstation and server GPUs, for inference and fine-tuning.
RTX 4090 and 3090.
Consumer GPUs, for batch jobs and workloads that tolerate interruption.
Observability
Prometheus · Grafana · OpenTelemetry
Provider health, job state and GPU telemetry in one view.
Alerts route to on-call, with a runbook for each alert group.
Security
Isolated containers · Signed images · Vault · Encrypted tunnels
Jobs run in isolated containers. Secrets are issued at run time, not stored on provider machines.
Provider onboarding
Every provider joins
the same way.
From agent install to listed capacity, with verification before any customer workload is placed.
- 01
Register
Provider account
- 02
Install
Onboarding agent
- 03
Verify
Hardware checks
- 04
Benchmark
Reference workloads
- 05
Admit
Capacity listed
- 06
Schedule
Customer jobs
- 07
Meter
Usage per job
- 08
Monitor
Health and heartbeats
How workloads are placed
Each job declares what it needs: GPU model, memory and region. The scheduler places it on a verified provider with free capacity, or queues it until one is available.
How the platform handles a provider going offline
Missed heartbeats mark a provider unhealthy and stop new placements on it. Interrupted jobs are requeued on other providers where the workload allows it.
How platform changes roll out
The control plane ships through GitOps with Argo CD. Agent updates reach providers in stages, so a faulty release touches only a small part of the network.
The outcome
Distributed supply.
A dependable service.
Customers reach GPU capacity through one interface. Providers join through one repeatable path.
A repeatable path in.
Providers move from agent install to listed capacity the same way every time.
Capacity that has been checked.
Customer workloads run only on hardware that has passed verification.
One record of usage.
The same metering serves customer billing and provider payouts.
Bring this capability to your team
Inference & GPU platforms
Explore Infer8 and the engineering that turns accelerators into a metered, multi-tenant service.
Explore the service Discuss your project
Next project / 03