What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
Choose a cloud GPU by sizing the workload first, not by picking the newest accelerator name. Define whether you are pretraining, fine-tuning, running batch inference or serving an online API; record model size, numerical precision, context length, batch or concurrency targets, and latency or throughput requirements. Then verify per-GPU memory, the way GPUs communicate inside and between hosts, CPU/RAM/storage balance, regional capacity and the complete hourly cost. For example, published configurations range from 80 GB HBM3 H100 GPUs to 141 GB HBM3e H200 GPUs, while an eight-GPU AWS configuration can provide up to 1,128 GB of aggregate HBM3e. Those figures describe capacity, not guaranteed application speed, so benchmark a representative workload before committing to a long reservation.
At a glance
| Name | Best for | Key spec | Free plan / price | Platforms | Checked |
|---|---|---|---|---|---|
| Google Cloud A2 | Smaller-model training and single-host inference | GPU memory and machine details vary by A2 shape | No universal free tier; region- and SKU-specific hourly pricing | Google Cloud Compute Engine | Nov 2026 |
| Google Cloud A3 / H100 | Large-model training and inference with mature H100 software support | 80 GB HBM3 per H100 GPU in published A3 examples | No universal free tier; confirm live regional price | Google Cloud Compute Engine | Nov 2026 |
| Google Cloud A3 Ultra / H200 | Memory-heavy models and fewer-way sharding | 141 GB HBM3e per H200 GPU in published examples | No universal free tier; confirm live regional price | Google Cloud Compute Engine | Nov 2026 |
| Google Cloud A4 / B200 | Latest-generation, large-scale model work | 180 GB HBM3e per B200 in the catalog | No universal free tier; confirm live regional price | Google Cloud Compute Engine | Nov 2026 |
| AWS P5 | Eight-GPU training and tightly coupled distributed workloads | Up to eight H100 GPUs and 640 GB aggregate HBM3; NVSwitch and up to 3,200 Gbps EFA | No universal free tier; check exact size, region and purchase option | Amazon EC2 | Nov 2026 |
| AWS P5e/P5en | Higher-memory, multi-GPU training and inference | Up to eight H200 GPUs and 1,128 GB aggregate HBM3e; NVSwitch and up to 3,200 Gbps EFA | No universal free tier; check exact size, region and purchase option | Amazon EC2 | Nov 2026 |
| Azure ND H100 v5 | High-end deep-learning training and scale-out jobs | Eight H100 GPUs, 640 GB aggregate GPU memory, NVLink and InfiniBand | No universal free tier; check exact SKU, region and term | Microsoft Azure Virtual Machines | Nov 2026 |
| Azure NCads H100 v5 | Applied-AI training and batch inference on one or two GPUs | One or two H100 NVL GPUs, 94 GB each | No universal free tier; check exact SKU, region and term | Microsoft Azure Virtual Machines | Nov 2026 |
This is a screening table, not a cross-cloud performance ranking. Google Cloud’s GPU catalog, AWS’s P5 documentation and Microsoft’s ND H100 v5 documentation describe different machine shapes, networks and availability conditions.
As an Amazon Associate I earn from qualifying purchases.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →What information should you collect before choosing a GPU?
Write a one-page workload specification before looking at instance catalogs. The same model can need radically different hardware for a short fine-tune, a high-concurrency API or a long pretraining run.
- Job type: pretraining, supervised fine-tuning, parameter-efficient fine-tuning, batch generation or interactive serving.
- Model and checkpoint: parameter count, architecture, number of layers and whether weights are dense or a mixture of experts.
- Numerical format: FP32, BF16, FP16, FP8 or an integer quantization. Training often retains optimizer and gradient states that inference does not.
- Sequence length: maximum and typical prompt or training-token length. Long contexts increase activation or key-value-cache memory.
- Batch and concurrency: tokens per second, requests per second, simultaneous sequences and acceptable queueing.
- Service objective: time to first token, per-token latency, job completion deadline or total tokens processed per dollar.
- Scale: one GPU, several GPUs in one host, or multiple hosts.
- Data path: dataset size, read rate, checkpoint frequency and whether data sits on local SSD or remote storage.
If these fields are unknown, a price comparison is premature. Measure or estimate them with a small representative run, then use the result to select capacity and a purchase term.
Popular on Amazon
- Ideal for Gifting
- Ideal for a bookworm
- Compact for travelling
How much GPU memory do you need for LLM inference?
Start with memory, because a configuration that cannot hold weights and runtime state cannot meet a latency target at any price. A rough lower bound for weights is parameter count multiplied by bytes per parameter: a 70-billion-parameter model needs about 140 GB just for BF16 weights, before buffers and the key-value cache. Eight-bit weights require roughly half that weight storage, and four-bit weights roughly one quarter, but implementation overhead and quality trade-offs remain.
Inference also needs memory for the KV cache, temporary activations, CUDA and framework workspaces, tokenizer or batching buffers, and sometimes a second copy during loading. Longer context and higher concurrency increase KV-cache demand. Leave headroom rather than planning to fill every byte; an apparently fitting model can fail when a second request arrives or when the serving framework reserves workspace.
Recommended Free Tools
Per-device HBM is more useful than an aggregate headline. Google Cloud lists 80 GB HBM3 for H100 examples, 141 GB HBM3e for H200 examples and 180 GB HBM3e for B200 examples in its current GPU catalog. An eight-GPU host therefore has 640 GB, 1,128 GB or 1,440 GB of nominal aggregate memory respectively, but that memory is distributed. You need tensor parallelism, pipeline parallelism or another sharding method to use it, and communication overhead rises as the shard count grows.
Quantization can make a model fit on fewer GPUs, yet it changes arithmetic support, output quality and sometimes throughput. Treat the quantized checkpoint, serving engine, context window and concurrency target as one configuration; do not select a GPU from weight size alone.
Which GPU features matter for LLM training?
Training uses memory for weights, gradients, optimizer states, activations and temporary communication buffers. Full-parameter training can require several times the weight footprint, while parameter-efficient methods reduce the trainable-state portion but still need memory for the base model and activations. Gradient checkpointing, micro-batching and sequence parallelism can lower peak memory at the cost of extra computation or communication.
Also popular on Amazon
After memory, compare useful compute in the precision your framework supports. Architecture generation names are not a guarantee of faster or cheaper training: kernel availability, compiler versions, collective libraries and model implementation can reverse a paper specification advantage. Ask whether your framework supports the GPU’s BF16, FP8 or other paths and whether the required drivers and container images are available on that provider.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →For distributed training, communication is a first-order constraint. Inside a host, NVLink, NVSwitch or an equivalent fabric allows faster all-reduce and parameter exchange than ordinary PCIe paths. Between hosts, inspect InfiniBand, Elastic Fabric Adapter (EFA) or the provider’s stated bandwidth and RDMA support. AWS describes P5 and P5e/P5en as NVSwitch-connected families with up to 3,200 Gbps EFA networking; Azure documents NVLink and InfiniBand for ND H100 v5. Those links are enablers, not throughput guarantees: the framework’s collective implementation and topology still determine scaling.
How do you match workload type to an instance shape?
| Workload | Primary sizing question | Hardware emphasis |
|---|---|---|
| Pretraining | Can the full training state and dataset pipeline fit while meeting the deadline? | Large HBM, high intra- and inter-host bandwidth, fast storage and reliable multi-node orchestration |
| Full fine-tuning | How many GPUs are required for weights, gradients, optimizer and activations? | Memory first, then compute and collective performance |
| Parameter-efficient fine-tuning | Can the base model, adapters and activations fit at the desired sequence length? | Enough HBM, moderate multi-GPU fabric and high data-loader efficiency |
| Batch inference | How many tokens per dollar can be generated in a fixed window? | Memory for weights and batches, sustained compute, storage for many prompts and checkpoint loading speed |
| Online inference | Can the service meet latency at peak concurrency? | KV-cache capacity, low communication latency, autoscaling options and headroom for traffic spikes |
Google’s guidance distinguishes general GPU machines from clustered strategies for larger, tightly coupled jobs; its AI Hypercomputer strategy guide is a useful topology check. Azure’s compute recommendations similarly point to ND-family machines for model training and NC or ND options for inference. These are starting points, not substitutes for sizing your model.
When is one host better than several?
Use one host when the model and training state fit and your software can exploit its local GPU fabric. A single host avoids inter-node coordination, placement failures and network charges, and often simplifies checkpointing. It is usually the sensible first deployment for a small fine-tune, a single-model API or batch work that fits in one machine.
Move to multiple hosts when memory, throughput or a completion deadline requires it. Before doing so, verify that the provider can place the requested hosts together, that the framework supports the required collective and RDMA path, and that checkpoints and failures can be recovered without restarting an excessively long job. A larger aggregate HBM number is useful only if your parallelism strategy can partition the model efficiently.
Also popular on Amazon
Compare the communication topology in the exact machine type, not just the family name. A provider may offer several sizes with different GPU counts, CPU ratios, network interfaces or local-disk layouts. Read the machine-level tables in Google Cloud’s GPU documentation and the corresponding AWS or Azure SKU pages before writing deployment code.
What do the major cloud GPU families offer?
Google Cloud
Google Cloud’s catalog lists A4/B200, A3 Ultra/H200, A3/H100 and A2/A100 options, with published memory, interconnect and workload descriptions. B200 entries list 180 GB HBM3e per GPU; H200 entries list 141 GB HBM3e; H100 entries list 80 GB HBM3; and A100 configurations are offered with 40 or 80 GB. Google positions later A-series families for large-scale model work and describes A2 as suitable for smaller-model training and single-host inference. Confirm the exact machine type, GPU count, host CPU/RAM, network and region because a family label is not a complete specification.
AWS
AWS P5 instances provide up to eight H100 GPUs and 640 GB of aggregate HBM3. P5e and P5en provide up to eight H200 GPUs and 1,128 GB of aggregate HBM3e. AWS documents NVSwitch GPU connectivity and up to 3,200 Gbps EFA networking for these families. Select the exact size and purchase option only after checking regional capacity, quotas and the serving or training framework you plan to run.
Azure
Microsoft’s ND H100 v5 is an eight-H100 family intended for high-end deep-learning training and tightly coupled scale-up or scale-out jobs, with NVLink and InfiniBand. NCads H100 v5 offers one or two H100 NVL GPUs with 94 GB each and is described for applied-AI training and batch inference. Azure’s compute guidance recommends ND machines for training and NC or ND for inference, but the model’s memory and latency requirements should determine the final SKU.
How should you compare CPU, RAM, storage and networking?
GPUs can sit idle while waiting for the rest of the instance. Check host CPU cores and system RAM for tokenization, data preprocessing, retrieval, request routing and checkpoint assembly. A high-throughput training loader may need several CPU workers and enough RAM to stage batches; an inference server may need RAM for multiple model replicas, tokenizer state or a large prompt queue.
Also popular on Amazon
Choose local SSD when you need fast checkpoint writes, dataset caching or predictable read latency. Use persistent or object storage for durability and sharing, but include startup downloads and network throughput in the schedule. Measure the data rate required to keep all GPUs busy and verify that the selected VM’s network interface and storage limits can sustain it. Include egress and inter-region transfer when data or users are outside the GPU region.
For online serving, also inspect load-balancer, autoscaling and regional placement behavior. A theoretically cheaper GPU can cost more if it forces cross-region traffic, has long provisioning times or leaves you paying for idle replicas.
How do you calculate the full cost?
Use a workload cost model rather than a GPU-hour headline:
- Estimate wall-clock runtime or serving hours at the target utilization.
- Multiply by the exact machine’s current regional hourly rate and purchase term.
- Add attached disks, snapshots, object storage, database or orchestration charges and data transfer.
- Include startup, checkpoint, retry and idle time. Distributed jobs often reserve every host for the full run.
- Divide by a useful unit such as completed training tokens, fine-tuned checkpoint, generated tokens or successful requests at the required latency.
Google publishes accelerator-optimized VM prices by machine and pricing option on its pricing page. AWS and Azure require the same check in their live pricing tools for the exact SKU, region and term. Prices, discounts, quotas and availability are account- and region-sensitive; the reviewed sources do not establish a universal cross-cloud price ranking.
Spot or preemptible capacity can reduce compute cost for checkpointable training, but interruption risk adds engineering and restart time. Reservations or capacity blocks can protect a deadline when available, yet they create commitment risk if your model or schedule changes. Treat scarce capacity and the cost of adapting code to another GPU family as part of the financial decision.
Also popular on Amazon
- It can be a gift option
- Comes with secure packaging
- Helpful in various ways
What is a practical step-by-step selection process?
- Write acceptance criteria. State model, precision, context, batch or concurrency, throughput, latency, deadline and maximum budget.
- Compute a memory floor. Sum weights plus gradients, optimizer states and activations for training, or weights plus KV cache and runtime buffers for inference. Add headroom.
- Choose the smallest viable topology. Prefer one host if it fits and meets throughput; otherwise define tensor, pipeline or data parallelism and the number of GPUs.
- Filter by interconnect. Require NVLink/NVSwitch-class intra-host links for tightly coupled sharding and verify InfiniBand, EFA or equivalent RDMA for multi-host training.
- Validate the host. Check CPU, RAM, local SSD, persistent storage limits, network bandwidth, driver versions and container or framework support.
- Shortlist regional SKUs. Confirm quotas, capacity, provisioning lead time and whether the exact machine is offered where your data and users are located.
- Run a representative benchmark. Use the real model, tokenizer, precision, context, batch or concurrency and serving engine. Record tokens per second, latency percentiles, memory headroom and cost per useful unit.
- Model failure and scaling. Test checkpoint recovery, host loss, autoscaling behavior and multi-node restart time before a production commitment.
- Select the purchase term. Compare on-demand flexibility with spot/preemptible interruption risk and reserved or capacity-block commitments.
- Recheck before launch. GPU catalogs, prices, quotas and regional inventory change; confirm them on the provider’s current pages on the day you provision.
What common mistakes make a cloud GPU choice fail?
- Adding GPU memory as if it were one pool: eight GPUs do not automatically behave like one 1,000-GB device. Your sharding plan and communication path must use the aggregate capacity.
- Choosing by generation name: a newer accelerator may be slower or more expensive for an unsupported kernel or precision path. Benchmark the actual software stack.
- Ignoring KV cache: a model can load successfully and then run out of memory when context or concurrent requests increase.
- Under-sizing the host: insufficient CPU, RAM, storage or input bandwidth starves expensive GPUs.
- Assuming advertised bandwidth equals application speed: collective algorithms, topology and message sizes determine distributed scaling.
- Using a family-level price: GPU count, host shape, region, disk, transfer and purchase term change the bill.
- Skipping quota and capacity checks: a technically suitable SKU is not useful if your account cannot obtain it before the deadline.
- Committing before a realistic run: vendor specifications describe hardware capability, not your model’s end-to-end throughput or latency.
How we picked the examples
The examples are drawn from current provider documentation and selected to show distinct memory capacities, GPU counts and communication topologies relevant to LLM training or inference. The criteria were workload fit, per-device memory, scale-up and scale-out interconnects, host and I/O considerations, and the need to verify regional price and availability. No independent performance ranking is implied.
What should you do immediately before provisioning?
Save the exact SKU, region, hourly rate, quota approval and machine specification in the run plan. Record the model revision, container image, driver and framework versions, parallelism settings, expected utilization and checkpoint location. Start with a cancellable pilot or short on-demand run, capture the benchmark metrics, and only then decide whether a reserved, capacity-block or interruption-prone purchase is justified.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallThe right cloud GPU is the least expensive configuration that demonstrably holds the model, meets the service objective and keeps its accelerators fed. Revisit the choice when model size, context, concurrency, software kernels, region or pricing changes.
Frequently Asked Questions
Can I run an LLM on a GPU with exactly the same memory as its weights?
Usually not. Runtime workspaces, CUDA allocations, activations and KV cache require additional memory, and serving systems need headroom for batching or concurrent requests. Leave capacity for the longest context and peak concurrency you will accept.
Is an H200 always faster or cheaper than an H100 for inference?
No. H200’s larger HBM capacity can reduce sharding or allow larger batches, but speed and cost depend on precision support, kernels, model architecture, utilization and regional pricing. Compare both with the same serving stack and traffic profile.
Should I use spot or preemptible GPUs for model training?
They can work for checkpointable jobs whose deadlines tolerate interruption. Calculate the expected restart and engineering cost, and verify that your provider offers enough replacement capacity in the required region before relying on them.
How do I estimate the number of GPUs for online serving?
Measure memory per replica at the target context and concurrency, then measure tokens per second and latency under representative traffic. Divide required throughput by sustainable per-replica throughput, round up for redundancy and traffic spikes, and include the communication overhead of any model sharding.
