Choose the workload before you choose the server. GPU memory and workload shape determine the useful GPU count; then CPU sockets, PCIe topology, host RAM, networking, storage, cooling and serviceability must be balanced around those GPUs. NVIDIA’s certified guidance (updated September 30, 2026) starts with two CPU sockets, at least six physical CPU cores per GPU, PCIe Gen5-or-later, host memory of at least twice aggregate GPU memory, and balanced placement across sockets and root ports. For multi-node inference, plan for at least 200 Gbps and, in larger designs, up to 400 Gbps per GPU.
A single eight-GPU server can be appropriate for a contained workload. If the job must scale beyond one chassis, buy repeatable nodes and a validated network pattern rather than adding servers one at a time. The sections below turn those requirements into a procurement process.
As an Amazon Associate I earn from qualifying purchases.
Start with the workload, not a GPU shopping list
“How many GPUs do I need?” has no universal answer. Training, fine-tuning, high-throughput inference and visualization stress different parts of a system. GPU count is only useful after you know the model’s memory footprint, concurrency target, data movement and whether the job must run across nodes.
Training
Training generally benefits from multiple synchronized GPUs and fast movement of gradients and data. NVIDIA documents balanced two-, four- and eight-GPU server configurations for deep-learning training. A larger count can shorten a run only when the model, software and input pipeline scale efficiently; otherwise, extra accelerators spend time waiting on CPU work, storage or communication.
#1 Best Overall
Fine-tuning
Fine-tuning may fit within fewer GPUs than full pre-training, but optimizer states, activation checkpoints and sequence length can make memory the limiting resource. Measure the complete fine-tuning job—not just the model’s parameter count—before selecting a card or chassis.
High-throughput inference
Inference sizing depends on concurrent requests, latency targets, batch size and model memory. A single node is often sufficient when the service can be contained in one chassis. Multi-node inference adds east-west traffic and makes network bandwidth a first-order capacity requirement.
Visualization and interactive workloads
Visualization can prioritize graphics features, display outputs and predictable interactive latency over distributed-training throughput. It may need fewer GPUs, but the chosen cards still have to fit the server’s power, cooling and PCIe constraints.
Recommended Free Tools
A practical sizing sequence
- Describe the job: training, fine-tuning, inference or visualization; record latency, throughput and concurrency targets.
- Determine memory demand: include model weights, framework overhead, activations, optimizer state, batching and headroom.
- Decide the deployment boundary: one server or a cluster of nodes.
- Select a validated GPU pattern: use a balanced two-, four- or eight-GPU design where applicable, then verify software support and interconnects.
- Balance the host: size CPU cores, RAM, PCIe lanes, NICs, NVMe, power and cooling around the selected GPU count.
GPU count and memory: the first hard constraint
GPU memory is usually the non-negotiable limit. A card with insufficient memory cannot be rescued by adding system RAM unless the software explicitly supports a workable offload strategy, which can change performance and operational complexity. Compare total available GPU memory, memory bandwidth, GPU-to-GPU interconnect and supported frameworks—not just the number of cards.
| Workload question | What to measure | Buying implication |
|---|---|---|
| Can the model and runtime fit on one accelerator? | Weights, activations, optimizer state, batch size and framework overhead | Choose GPU memory capacity before maximizing GPU count. |
| Must GPUs communicate during the job? | Gradient, activation or request-transfer volume | Prioritize GPU interconnect and a balanced PCIe topology. |
| Does demand exceed one chassis? | Concurrent users, throughput target and growth rate | Evaluate multi-node networking and a repeatable cluster unit. |
| Is the workload interactive? | Latency consistency, display or visualization requirements | Favor predictable cooling, power delivery and serviceability. |
NVIDIA’s certified configurations cover two, four and eight GPUs for documented inference and deep-learning training platforms. Treat those as validated starting patterns, not a promise that every model scales linearly. A smaller, well-balanced server can outperform a larger but poorly connected one.
CPU platform requirements for multi-GPU servers
NVIDIA’s certified starting point is a platform with at least two CPU sockets, PCIe Gen5-or-later capability, a minimum 2.0 GHz base clock and at least six physical CPU cores per GPU. These are platform requirements for the documented configurations, not a universal guarantee for every application.
| Requirement | Certified guidance | Why it matters |
|---|---|---|
| CPU sockets | At least two | Provides the socket and memory-channel topology needed to feed multiple accelerators and distribute PCIe devices. |
| Physical cores | At least six per GPU | Leaves CPU capacity for data preparation, orchestration, networking and storage alongside accelerator work. |
| Base clock | At least 2.0 GHz | Sets a documented floor for host-side work; actual performance also depends on architecture and sustained behavior. |
| PCIe generation | Gen5 or later | Provides the link bandwidth expected by the certified GPU and I/O arrangements. |
Translate the core rule into a planning floor by multiplying six physical cores by the number of GPUs, then add capacity for operating-system services, data loaders, containers, monitoring and failover. Do not count hyperthreads as physical cores. Confirm the server’s actual lane map: a processor may advertise many lanes while the chassis routes only a subset to GPU slots.
Rank #2
- Ideal for Gifting
- Ideal for a bookworm
- Compact for travelling
Host memory: apply the two-times rule correctly
NVIDIA specifies minimum system memory equal to twice the aggregate GPU memory. If a server has N GPUs and each has M GB of memory, the host-memory floor is 2 × N × M GB. This is a minimum sizing rule, not a workload-specific performance guarantee.
Install that capacity evenly across CPU sockets and memory channels. Filling the available memory slots can improve bandwidth, provided the modules are supported by the platform. Uneven population can leave one socket or channel starved even when the total capacity looks adequate.
Memory compatibility checks
- Use the server manufacturer’s qualified ECC RDIMM or MRDIMM list; capacity alone does not establish compatibility.
- Match the permitted DDR5 generation, speed, rank and module population rules for the exact motherboard and processor combination.
- Verify that both sockets receive balanced capacity and that the BIOS recognizes the intended channel population.
- Reserve practical headroom for data staging, containers, caching and growth instead of treating the minimum as a target.
HPE’s current ProLiant AI portfolio lists configurations from 768 GB to 8 TB of maximum memory, depending on model. That range illustrates why the exact server model and module qualification matter more than a generic “AI server RAM” label.
PCIe topology and lane placement
Multi-GPU performance depends on where devices attach, not simply on how many slots appear in the chassis. Balance GPUs across CPU sockets and root ports. When possible, place NICs and NVMe drives under the same PCIe switch or root complex as the GPUs they serve. NVIDIA recommends PCIe Gen5 switches for the documented configurations.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWhat to request from a vendor
- A slot-to-socket map showing every GPU, NIC, NVMe device and PCIe switch.
- The number of electrically available lanes per GPU slot, not just the mechanical slot width.
- Whether any slot shares bandwidth with another GPU, NIC or storage device.
- NUMA guidance for pinning CPU threads, memory and network queues to the nearest GPU.
- Supported firmware settings for peer-to-peer GPU transfers and I/O virtualization.
A product such as the NVIDIA RTX PRO 6000 Blackwell Server Edition GPU requires a PCIe Gen5 x16-or-better slot for optimized connectivity. Confirm chassis support, auxiliary power, cooling clearance, firmware and the seller’s current availability before placing an order.
Networking: when 200 GbE or 400 GbE is justified
Single-node workloads normally do not need high-speed networking between nodes. They still need a suitable north-south connection for users, storage and management. Multi-node inference requires deliberate east-west design: NVIDIA’s certified baseline is a minimum 200 Gbps NIC, with guidance up to 400 Gbps per GPU.
“Per GPU” is important. It does not mean one 400 Gbps port automatically satisfies an eight-GPU cluster. Count the adapters, ports and switching capacity required by the chosen topology, then verify that the server has enough PCIe lanes and physical space for them.
NVIDIA’s C-G-N-B notation
NVIDIA describes reference architectures as C-G-N-B: CPU sockets, GPUs, network adapters and average east-west bandwidth per GPU in GbE.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →| Notation | Meaning | Use in planning |
|---|---|---|
| 2-8-5-200 | Two CPU sockets, eight GPUs, five network adapters, 200 GbE per GPU | Baseline-style eight-GPU pattern for substantial multi-node traffic. |
| 2-8-9-800 | Two CPU sockets, eight GPUs, nine network adapters, 800 GbE per GPU | High-bandwidth reference pattern for exceptionally communication-heavy designs. |
| 2-4-5-800 | Two CPU sockets, four GPUs, five network adapters, 800 GbE per GPU | High-bandwidth four-GPU pattern. |
The bandwidth figures in these examples describe average east-west bandwidth per GPU in the reference notation. They are not a substitute for a workload benchmark or a switch design. Check oversubscription, port count, cable and transceiver compatibility, fabric management and failure behavior.
North-south versus east-west traffic
- North-south: client requests, storage access, image distribution and management traffic entering or leaving the cluster.
- East-west: GPU-to-GPU and node-to-node communication inside the cluster.
Size the east-west fabric for synchronized training or distributed inference, then separately verify that user-facing and storage links can sustain peak demand. A cluster can have adequate external connectivity and still bottleneck internally.
One large server or a cluster?
Choose one server when
- The model and target throughput fit within one chassis.
- Operational simplicity and a single failure domain are more valuable than horizontal expansion.
- The facility can supply the server’s power, cooling and rack space.
- There is no requirement to distribute inference replicas or training workers across nodes.
Choose a cluster when
- The model, concurrency or throughput target exceeds one server.
- You need independent failure domains, rolling maintenance or capacity growth.
- The software stack supports distributed execution and the facility can operate a high-speed east-west fabric.
- You can buy nodes in a repeatable pattern instead of mixing unmatched generations.
Clustering adds switches, NICs, optics or cables, fabric management, rack power and cooling, and more failure modes. It is justified when the workload or growth plan requires those capabilities—not merely because a larger GPU count looks attractive on a specification sheet.
Use scalable units for predictable growth
NVIDIA defines a scalable unit as a repeatable building block for expansion. For the RTX PRO and HGX AI Factory families, one scalable unit is four nodes; the minimum deployment is one four-node unit, with expansion in four-node increments.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →That pattern keeps compute, network ports, power distribution and cooling additions aligned. Before buying, document the unit’s GPU count, CPU and memory configuration, NIC pattern, switch ports, rack units, expected power draw and cooling method. If a later purchase changes any of those elements, treat it as a new validation exercise rather than assuming seamless compatibility.
Cooling, power and rack planning
Accelerator specifications do not tell you whether a facility can run the server continuously. Cooling method, rack density and power delivery belong in the initial design.
Rank #4
Air versus liquid cooling
Air-cooled systems simplify facility requirements but can impose limits on density and ambient conditions. Liquid-cooled systems can support dense GPU configurations, but they require compatible rack infrastructure, plumbing or cooling-distribution equipment, leak controls and trained service procedures. Compare the complete facility design, not just the server’s advertised GPU count.
Power delivery
- Confirm circuit capacity, connector type, redundancy and rack-level distribution.
- Verify that peak GPU and CPU demand, startup behavior and expansion plans fit the facility envelope.
- Include networking, storage and cooling equipment in the rack budget.
- Ask how a failed power supply, pump or fan is replaced without taking the entire workload offline.
Remote management and security requirements
NVIDIA’s certified guidance calls for Redfish 1.0 or later for remote management and TPM 2.0 secure boot. These controls reduce the need for physical access and help establish a measurable firmware and boot chain.
Procurement questions
- Can administrators inventory hardware, update firmware and collect health data through Redfish?
- Does TPM 2.0 secure boot integrate with the organization’s operating-system and attestation policy?
- Are GPU, NIC, switch and BIOS firmware versions tested as a supported combination?
- What is the vendor’s lifecycle policy for security fixes and out-of-band management software?
Remote management is especially valuable in a cluster, where a small firmware mismatch can create inconsistent behavior across otherwise identical nodes.
Local NVMe placement and storage
NVIDIA recommends at least one NVMe drive per CPU socket, with at least 1 TB. Place drives with the same PCIe locality considerations used for GPUs and NICs. Local NVMe can stage datasets, containers, checkpoints and logs without forcing every operation through a remote storage fabric.
Capacity planning should distinguish between the operating system, container images, active datasets, checkpoints, temporary preprocessing files and retained logs. Confirm endurance, encryption support, hot-swap behavior and the server’s approved drive list; a nominally compatible drive may still lack validated firmware or service support.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Current enterprise server examples
HPE describes a purpose-built portfolio for AI training, tuning and inference. Its examples show how CPU, GPU and cooling choices change the platform:
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minute| System | GPU configuration | CPU configuration | Cooling |
|---|---|---|---|
| HPE XD685 | Eight NVIDIA B200, NVIDIA B300 or AMD MI355X GPUs | Two fifth-generation AMD EPYC processors | Liquid cooling |
| HPE XD690 | Eight NVIDIA B300 GPUs | Two Intel Xeon 6 processors | Air cooling |
HPE’s broader ProLiant AI portfolio lists NVIDIA RTX PRO 4500 and 6000 Blackwell options, AMD EPYC and Intel Xeon 6 configurations, and maximum memory capacities from 768 GB to 8 TB depending on model. Treat these as platform examples, not interchangeable parts lists: every GPU, processor, memory module, NIC, riser and cooling option must be validated for the exact chassis.
Best Value
- It can be a gift option
- Comes with secure packaging
- Helpful in various ways
A finance-aware total-cost framework
The purchase price is only one line in an enterprise AI server decision. Build a cost model with separate one-time and recurring categories so a seemingly inexpensive node does not create an unaffordable operating burden.
One-time costs
- Servers, GPUs, CPUs, memory, NVMe drives and NICs.
- Switches, optics, cables, rack hardware and power-distribution equipment.
- Liquid-cooling distribution or facility modifications where required.
- Deployment, integration, software qualification and data-center installation.
Recurring costs
- Electricity for compute, networking and cooling.
- Support contracts, spare parts, firmware lifecycle work and specialist labor.
- Software subscriptions or enterprise support required by the workload.
- Rack space and facility capacity reserved for growth.
Compare alternatives over the period you expect to operate them. A denser liquid-cooled system may reduce rack count while increasing facility complexity; several smaller air-cooled nodes may simplify replacement while consuming more rack space and network ports. Use measured power and utilization from a proof-of-concept when available, and label every estimate with its assumptions.
Common buying mistakes and how to correct them
Buying GPUs before checking the chassis
Symptom: the cards fit physically but exceed slot power, auxiliary-cable, thermal or firmware limits. Correction: obtain the vendor’s supported GPU matrix, power budget, airflow or liquid-cooling specification and BIOS version before ordering.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Counting threads instead of physical cores
Symptom: the server appears to meet a CPU target but data loading and orchestration contend with the workload. Correction: apply the six-physical-cores-per-GPU floor and reserve additional cores for host services.
Installing enough total RAM but not enough bandwidth
Symptom: capacity is adequate while throughput remains poor or one NUMA node is saturated. Correction: distribute modules evenly across sockets and channels and follow the qualified population rules.
Putting every GPU behind one root complex
Symptom: peer transfers or NIC traffic bottleneck despite Gen5 links. Correction: rebalance GPUs across sockets and root ports, and place related NICs and NVMe devices near their serving GPUs.
Underbuilding the east-west fabric
Symptom: multi-node inference or training spends time waiting on communication. Correction: size NICs and switching for the per-GPU bandwidth target, check oversubscription and confirm the server has lanes and slots for the required adapters.
Expanding with unmatched nodes
Symptom: mixed firmware, GPU or network generations complicate scheduling and support. Correction: define a scalable unit and add capacity in the validated increment whenever possible.
Final procurement checklist
- Write the workload, model memory requirement, latency or throughput target and expected growth.
- Select a supported two-, four- or eight-GPU pattern, or document why a different pattern is required.
- Verify two CPU sockets, PCIe Gen5-or-later capability, base-clock requirement and physical-core allocation.
- Calculate host-memory capacity from aggregate GPU memory, then populate sockets and channels symmetrically.
- Review the complete PCIe topology, including GPU, NIC, NVMe and switch placement.
- Choose single-node networking or a multi-node fabric; for the latter, validate at least 200 Gbps and the required per-GPU target.
- Confirm NVMe capacity and locality, Redfish 1.0-or-later management and TPM 2.0 secure boot.
- Approve air or liquid cooling, rack density, power circuits, redundancy and service procedures with the facility team.
- Model acquisition, installation, energy, cooling, support and expansion costs over the planned operating period.
- Require a vendor-provided configuration bill of materials and written support matrix before issuing a purchase order.
Bottom line
Buy a balanced system, not the largest GPU count you can fit. Start with GPU memory and workload behavior; enforce the CPU, host-memory and PCIe rules; build networking around east-west traffic; and treat cooling, security, serviceability and expansion as equal procurement criteria. One validated server is the right answer when the workload fits. When it does not, repeatable four-node scalable units and a properly provisioned fabric provide a more predictable path than improvised growth.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
