Introduction

In modern computing, GPUs have become essential for intensive workloads such as machine learning, deep learning, and scientific computing. The NVIDIA H100, one of the leading GPUs on the market, delivers outstanding performance with 80 GB of HBM3 memory. To fully unlock the H100 in a complex Kubernetes environment, you need to understand how to integrate RDMA (Remote Direct Memory Access) into your Pod configuration.

This article walks you through every aspect of using H100 GPUs with RDMA support in a Kubernetes cluster. From initial setup to troubleshooting, it covers what you need for a successful deployment and optimal performance.

RDMA (Remote Direct Memory Access) lets a computer access memory on a remote machine without involving the host CPU. RDMA becomes essential when your workload uses 16 GPUs or more. Lintasarta nodes provide 8 H100 GPUs per node, so the common setups are:

• Single-node scenario (1-8 GPUs) — All GPUs run in one node, with communication over internal NVLink.

  • Example: Fine-tuning a 7B-13B parameter model

  • RDMA: NOT REQUIRED

  • Pod configuration: Request only nvidia.com/gpu and skip rdma/ib_shared_device

• Multi-node scenario (16+ GPUs) — GPUs are distributed across 2 or more nodes.

  • Example: Training a 70B-175B parameter model with distributed training

  • Typical setup: 2 nodes (16 GPUs total) or 4 nodes (32 GPUs total)

  • RDMA: REQUIRED for inter-node communication

  • Pod configuration: Request both nvidia.com/gpu and rdma/ib_shared_device