This guide is for operators connecting GPU servers to a managed GPU cloud. The control plane runs on the hosted.ai GPUaaS platform and is operated for you. Your job is to stand the nodes up to spec. They're onboarded as workers, start serving tenants, and the platform scales as you add nodes. The whole thing can be white-labelled under your own brand.
The managed Admin/User panel sits separately from the GPU nodes and connects to each region as hardware is onboarded. Within a region, one node can run the controller, worker and storage roles together; additional nodes join as workers. The diagram below is a reference pattern. Final network design is aligned with your standards during deployment.
Reference pattern adapted from the hosted.ai GPUaaS architectural reference. Addresses shown are illustrative; your node, pod and service ranges are confirmed during deployment.
A node can carry one or more roles. Your first node in a region typically carries all three; every node after it joins as a worker and adds GPU capacity to the pool.
Runs the in-region control functions and talks to the Admin/User panel over the management network, handling node management, scheduling and health reporting.
Hosts the actual containerised GPU workloads. Every additional node is a worker, so more nodes means more GPU capacity for tenants. Workers are added as demand grows.
Presents persistent block storage to GPU workloads. Storage can be local to the node or presented over the network, depending on your architecture and target workloads.
The platform needs connectivity between the control plane and your GPU nodes, plus external connectivity for customer workloads. These are typical reference patterns; final design is aligned with your standards during deployment.
Private connectivity between the control plane and your GPU nodes, used for node management, scheduling and health reporting. Typically a standard VLAN or equivalent segmentation.
Carries customer access to workloads and exposed services. Can be backed by public IPs, NAT, load balancers or private connectivity. Outbound access for image and model pulls follows your security policies.
For multi-node or latency-sensitive workloads, high-bandwidth interconnects such as NVLink, InfiniBand or RoCE can be supported where available.
192.168.X.X/24192.168.100.0/24 overlay (predefined)10.96.0.0/12 (predefined)122.111.222.111Persistent storage may be local or presented over the network, depending on your architecture and the workloads you intend to run.
1 TB or larger NVMe preferred (SSD optional) for the OS and platform services.
GPU workload scratch disk: fast, transient space for the running container.
GPU persistent block storage, local to the node or network-presented.
Depending on the deployment model and target workloads, the platform scales in several directions, all under the same control plane.
GPUaaS worker nodes are added as demand for GPUs grows. Each new node to spec joins the pool and starts serving tenants.
Horizontal scaling and multiple regions are supported. The same panel manages nodes in eu-west-2, eu-west-4 and beyond.
Low-latency, multi-node, multi-GPU workloads are supported over NVLink and InfiniBand where the fabric is in place.
Dedicated hardware tenant clusters can be carved out for customers that need isolated, reserved capacity.
GPU sizing and oversubscription work two ways. The right one depends on whether your hardware specs are still flexible or already fixed. Either way, the final ratios are agreed jointly before production tenants go live.
Define a target oversubscription strategy upfront, from conservative to aggressive GPU sharing, and we work backward from it.
Use when specs are still being finalised, or you're designing a new GPU platform from scratch.
Supply the final node specs and we validate against them.
Use when hardware is already procured or in delivery and the goal is to maximise utilisation without compromising service quality.
The control plane is already in place. Onboarding a node is a collaborative sequence between your team and ours; we align the network design to your standards as we go.
Ubuntu 24.04, root NVMe, GPUs in place, and bonded networking (LACP) with private and public VLANs ready.
Send node details and target use cases. We confirm IP ranges, the oversubscription path (A or B) and the role mix.
We attach the node to the control plane over the management network and bring up the worker and storage roles.
We confirm operational limits, then enable tenants. Ongoing node- and cluster-level operations run with our support.
Send those over and the onboarding team comes back with confirmed network ranges, a sizing path, and a deployment window for your region.