# GPU Infrastructure

Sunrise Cloud provides NVIDIA H100 GPU acceleration for AI and high-performance workloads, delivered on HGX nodes (8×H100 per node) in our Swiss data centers. This page explains how GPUs are exposed to tenants, how to choose a deployment model when provisioning, and how to schedule GPU workloads.

!!!info Deployment models at provisioning
When you add a GPU subscription in the [Sunrise Portal](https://portal.ai.sunrise.ch), choose **container GPU** (default) or **VM passthrough** (vfio-pci). Container GPUs use the `nvidia.com/gpu` resource for pods. VM passthrough dedicates the GPU to virtual machines on separate GPU nodes, enable it at provisioning time; see [GPU passthrough for VMs](#gpu-passthrough-for-vms) below.
!!!

---

## Scheduling GPU workloads (containers)

You do **not** need a `nodeSelector` or node label to place GPU workloads. Tenants cannot list nodes at tenant scope, and you don't have to. Instead, request the GPU resource in your pod spec and the Kubernetes scheduler will place the workload on a node that has the requested GPUs available.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: gpu-workload
  namespace: my-project
spec:
  containers:
    - name: cuda-container
      image: nvidia/cuda:12.4.1-base-ubi9
      resources:
        limits:
          nvidia.com/gpu: 1   # number of H100 GPUs requested
```

| You want | What to do |
|---|---|
| Place a workload on a GPU node | Request `nvidia.com/gpu` in `resources.limits`, no label or `nodeSelector` needed |
| Multiple GPUs in one container | Increase the `nvidia.com/gpu` value (up to 8 per HGX node) |
| Discover node labels yourself | Not required, and not possible at tenant scope, rely on the resource request instead |

!!!info GPUs for VMs use a different resource
The `nvidia.com/gpu` resource applies to **container** workloads. To use GPUs inside a **VM**, your namespace must instead be granted **passthrough GPUs**, which is a different resource type, see below.
!!!

---

## GPU deployment models

### Shared / container GPUs (available now)

Today, GPUs are exposed to tenants as the standard `nvidia.com/gpu` container resource. The NVIDIA GPU Operator runs on the bare-metal nodes (installed and managed by Sunrise) and advertises GPUs to the scheduler. This is the supported and tested path for AI training and inference workloads.

### GPU passthrough for VMs

**VM passthrough** (vfio-pci) dedicates GPU nodes to virtual machines, not shared with container workloads. You can **enable passthrough when provisioning** a GPU subscription in the Sunrise Portal. Passthrough uses a different resource type and requires the NVIDIA operator to be loaded differently on dedicated GPU nodes, separate from container GPU workloads.

| Aspect | Status |
|---|---|
| Container GPUs (`nvidia.com/gpu`) | Available, default when provisioning |
| GPU passthrough to VMs (vfio-pci) | Available, enable at GPU provisioning in the portal |

!!!info Choosing at provisioning
Select **container GPU** for OpenShift pods and AI workloads that request `nvidia.com/gpu`. Select **VM passthrough** when you need a dedicated GPU inside a KubeVirt virtual machine. The choice is made when you add the GPU to your Capacity Pool, you do not need a separate enablement request.
!!!

---

## permittedHostDevices and deviceNames (KubeVirt)

KubeVirt VM passthrough requires the H100 devices to be registered in the HyperConverged CR's `permittedHostDevices` with a `deviceName` that you then reference in the VM spec. The HyperConverged CR is not readable at tenant scope.

When you enable passthrough at GPU provisioning, Sunrise configures the dedicated GPU nodes and registers the permitted host devices for your namespace. The supported `deviceNames` and the corresponding VM spec snippet are provided as part of setup. If you need help with your VM spec, [open a service ticket](https://support.ai.sunrise.ch/).

---

## NVLink and NVSwitch topology

GPU interconnect today is validated for **container** workloads. Intra-node GPUs on an HGX node are connected via NVSwitch/NVLink; an NVLink-Network fabric spanning nodes (e.g. connecting ~100 GPUs to a single VM) has **not been tested or made available for VM workloads**.

| Question | Answer |
|---|---|
| Is interconnect tested for containers? | Yes, this is the supported path today |
| Multi-node NVLink fabric for a single VM | Not currently tested or supported |
| Need cross-node GPU topology for VMs | Requires a joint test with our team, [contact us](https://support.ai.sunrise.ch/) |

!!!info VM GPU sizing
VM GPU sizing (single VM per 8-GPU node vs. multi-node fabric) may be determined case by case for passthrough workloads. If your architecture depends on a specific NVLink scope, [contact us](https://support.ai.sunrise.ch/) so we can validate it together before you design around it.
!!!

---

## In-VM NVIDIA driver and CUDA responsibility

| Workload type | Who provides the driver / CUDA |
|---|---|
| **Containers** | The NVIDIA GPU Operator, installed and managed by Sunrise at the bare-metal level, provisions the driver. You build on a CUDA base image and do **not** install host drivers yourself. |
| **Passthrough VMs** | You install the **NVIDIA driver + CUDA 12 inside the guest yourself**. These VMs run on dedicated GPU nodes configured when you enable passthrough at provisioning. |

!!!info Why VMs differ
The GPU Operator is installed at the bare-metal level by Sunrise for container workloads. Enabling passthrough requires changing that operator on the affected nodes and placing passthrough VMs on separate GPU nodes, which is why, for VMs, the in-guest driver and CUDA stack is the tenant's responsibility.
!!!

---

## Networking and inter-VM / inter-pod traffic

OpenShift ships **without** a cluster-level default-deny `NetworkPolicy`. East-west traffic between your VMs and pods is therefore **allowed by default**, including service ports such as MariaDB on TCP 3306 and DNS.

Because there is no default deny, you should add your **own** `NetworkPolicy` objects to block any services that must not be reachable from outside your namespace.

```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: my-project
spec:
  podSelector: {}
  policyTypes:
    - Ingress
```

!!!warning Protect exposed services yourself
With no default deny in place, unprotected services are reachable across namespaces. Define explicit `NetworkPolicy` rules for anything sensitive. A platform-wide default-deny is planned for the future, but because of the potential impact on existing tenants it will be introduced gradually, do not rely on it today.
!!!

---

## Monitoring GPU workloads

OpenShift's built-in monitoring stack (Prometheus/Grafana) is **not currently exposed to tenant namespaces**, and you cannot list `ServiceMonitor` objects or the `gpu-operator` namespace at tenant scope.

| Need | Current option |
|---|---|
| Prometheus/Grafana for your namespace | **Self-host** your own monitoring stack today |
| GPU Operator DCGM-exporter dashboard | Not exposed to tenants, self-host DCGM/Grafana for GPU metrics |
| Shared, platform-provided monitoring | Planned, a shared tenant-facing instance is on the roadmap |

!!!info Want early access to shared monitoring?
We plan to make a shared monitoring instance available to tenants. If observability for GPU workloads is important to you now, [contact us](https://support.ai.sunrise.ch/) and we can check whether the team can onboard you as an early (sponsor) user.
!!!

---

## Roadmap summary

| Capability | Status |
|---|---|
| Container GPUs (`nvidia.com/gpu`) | Available now, default at provisioning |
| GPU passthrough to VMs (vfio-pci) | Available now, enable at GPU provisioning |
| Multi-node NVLink fabric for VMs | Requires joint testing; not generally available |
| Platform default-deny NetworkPolicy | Planned, phased rollout |
| Shared tenant monitoring (Prometheus/Grafana + DCGM) | Planned; early access on request |

---

## Related Pages

- [IaaS Capacity and Node Provisioning](./iaas-capacity-and-node-provisioning.md)
- [OpenShift Architecture Overview](./openshift-architecture-overview.md)
- [Namespaces, Quotas and RBAC](./namespaces-quotas-rbac.md)
- [Access Your OpenShift AI](../getting-started/access-your-openshift-ai.md)
- [Confidential Computing](../confidential-computing.md)