Share GPUs with consumable capacity

Kubernetes DRA consumable capacity lets multiple independent ResourceClaim objects allocate the same physical GPU or MIG device. The NVIDIA driver exposes this capability through its ConsumableShares feature gate and consumableShares Helm value. It marks supported devices as multi-allocatable and publishes a capacity policy; the Kubernetes scheduler then tracks how much capacity each claim consumes.

Use consumable capacity when separate pods need independent claims but can safely co-locate on one device. This differs from time-slicing and MPS, where multiple containers reference an allocation with a driver-specific sharing configuration.

Feature status

ConsumableShares is the NVIDIA driver’s Alpha feature gate and is disabled by default. It must be enabled together with a non-disabled consumableShares mode. The corresponding upstream Kubernetes feature is DRA consumable capacity, controlled by the DRAConsumableCapacity feature gate.

Refer to the feature gates reference for all available gates.

Prerequisites

• Install the DRA Driver for NVIDIA GPUs. Refer to Installation.
• Use Kubernetes v1.34 or later. On Kubernetes v1.34 and v1.35, enable the upstream DRAConsumableCapacity feature gate on the kube-apiserver, kube-controller-manager, kube-scheduler, and kubelet. Kubernetes v1.36 and later enables that gate by default. Refer to Kubernetes consumable capacity.
• Use the gpu.nvidia.com or mig.nvidia.com DeviceClass. The driver’s consumable-capacity implementation does not support VFIO passthrough devices.
• Do not use an MPS configuration on claims that use a shared device.

Enable consumable capacity

Enable the driver’s consumable-capacity implementation and choose one driver-specific accounting mode for all GPU kubelet plugins in the Helm release. This example selects memory-based accounting:

helm upgrade dra-driver-nvidia-gpu oci://registry.k8s.io/dra-driver-nvidia/charts/dra-driver-nvidia-gpu \
  --namespace dra-driver-nvidia-gpu \
  --set featureGates.ConsumableShares=true \
  --set consumableShares=memory \
  --set gpuResourcesEnabledOverride=true

The Helm upgrade rolls the GPU kubelet plugin pods so they republish their ResourceSlice objects with allowMultipleAllocations: true and the selected capacity request policy.

To disable the feature, set consumableShares=disabled or clear the value. You can then disable the ConsumableShares feature gate.

Choose a capacity accounting mode

The mode is a cluster-administrator setting applied to supported devices on every node managed by the Helm release. Workload authors express their needs in the capacity.requests map of a ResourceClaim or ResourceClaimTemplate.

Memory mode

With consumableShares=memory, request the amount of GPU memory that the scheduler should reserve for each claim:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-4gi
spec:
  spec:
    devices:
      requests:
      - name: gpu
        exactly:
          deviceClassName: gpu.nvidia.com
          capacity:
            requests:
              memory: 4Gi

Each pod that references this template gets its own claim. Kubernetes can allocate multiple claims to one GPU while their combined requested memory fits within the device’s advertised memory capacity. If capacity.requests.memory is omitted in memory mode, the claim consumes the full memory capacity and no other memory-consuming claim can be allocated to that device.

Unlimited mode

With consumableShares=unlimited, omit capacity from the device request:

devices:
  requests:
  - name: gpu
    exactly:
      deviceClassName: gpu.nvidia.com

The claim consumes zero capacity by default. The scheduler can therefore place an unbounded number of these claims on the same device, subject to its other scheduling constraints.

Fixed share count

Set consumableShares to a positive integer to publish a fixed number of shares. For example, consumableShares=4 lets four claims that use the default of one share co-allocate on a device. To consume more than the default, add a shares request:

capacity:
  requests:
    shares: 2

Verify shared allocations

Inspect the published device policy:

kubectl get resourceslices -o yaml

For supported devices, verify that allowMultipleAllocations is true and that the memory or shares capacity contains the expected requestPolicy.

After creating workloads, inspect their claims:

kubectl get resourceclaims -A -o yaml

For a shared allocation, the allocation result records a shareID and its consumedCapacity. Example manifests for all three modes are available under demo/specs/consumable-shares/.

Limitations and considerations

• Capacity is scheduling accounting, not runtime isolation. A memory value reserves scheduler-visible capacity but does not limit a process’s CUDA memory use. Workloads sharing a device can affect each other’s memory use and performance.
• MPS is not supported while the driver’s consumable-capacity mode is active. The driver rejects an MPS-configured claim in this configuration.
• VFIO passthrough is not supported. The driver implements consumable capacity only for full GPUs and static or dynamic MIG devices.
• Claims must use matching device configurations. If a device is already prepared for another shared claim, a new claim with a conflicting GpuConfig or MigDeviceConfig fails to prepare.
• Unlimited means unbounded scheduler placement. Because a claim without a memory request consumes zero capacity in unlimited mode, use this mode only when the workloads can tolerate unrestricted contention.