Google Cloud is the lowest-cost option in the available like-for-like T4 comparison, but it is not universally the cheapest way to run inference. The April 2026 on-demand table lists a T4 GPU at $0.35 per hour on Google Cloud, compared with $0.53 on AWS and $0.56 on Azure. Those are accelerator comparisons, not complete application bills. Managed services charge by model tokens, while self-hosted deployments charge for GPUs plus machines, storage, networking and operations. The cheapest choice changes with model, precision, token mix, region, latency, utilization, interruption tolerance and data-residency requirements.
The short answer: cheapest depends on the inference model
There are two fundamentally different ways to buy inference:
As an Amazon Associate I earn from qualifying purchases.
- Managed model APIs: You send requests to a catalog model and pay primarily for input and output tokens. Bedrock, Microsoft Foundry and Vertex AI expose different models, rates and billing meters, so their prices cannot be ranked from a single GPU-hour table.
- Self-hosted inference: You operate a model on virtual machines with attached GPUs. The accelerator-hour rate is visible, but the total also includes the host machine, disks, network traffic, monitoring, orchestration, support and time when the GPU is idle.
For self-hosting, the April 2026 comparison gives Google Cloud the lowest on-demand T4 price at $0.35 per GPU-hour. AWS is $0.53 and Azure is $0.56. Google Cloud’s own published T4 figures list $0.35 per accelerator-hour on demand, $0.22 with a one-year commitment and $0.16 with a three-year commitment. It also describes typical Spot discounts of 60–91% for most machine types and GPUs; Spot prices are dynamic and can change when capacity or market conditions change.
These numbers answer only a narrow question: which provider’s listed T4 accelerator is cheapest under those pricing assumptions? They do not establish a universal winner for LLM APIs, a different GPU, a particular region or a production service with a strict uptime target.
#1 Best Overall
- Video/Sound Cards
- Passive Cooling
What the published GPU comparison actually tells you
| Provider or source | GPU or deployment | Price | Qualification |
|---|---|---|---|
| Google Cloud | NVIDIA T4, on demand | $0.35 per GPU-hour | Official GPU pricing page accessed 2026-10-18 |
| Google Cloud | NVIDIA T4, one-year commitment | $0.22 per GPU-hour | Official GPU pricing page; commitment pricing |
| Google Cloud | NVIDIA T4, three-year commitment | $0.16 per GPU-hour | Official GPU pricing page; commitment pricing |
| Google Cloud | NVIDIA L4, on demand | $0.72 per GPU-hour | Comparative on-demand table for 2026 |
| AWS | NVIDIA T4, on demand | $0.53 per GPU-hour | Comparative on-demand table for 2026 |
| Azure | NVIDIA T4, on demand | $0.56 per GPU-hour | Comparative on-demand table for 2026 |
| Three hyperscalers | H100, A100 and V100 Spot examples | $1.95–$12.29 per GPU-hour | Range reported for August 2026; GPU and provider vary |
The T4 figures are useful only when the workload can run on a T4 at the required precision, memory footprint, throughput and latency. A newer accelerator may cost more per hour but process enough requests to reduce cost per token. Conversely, a cheap T4 can be wasteful if it cannot meet peak demand and forces you to run several replicas.
As a simple capacity illustration, one T4 running continuously for 730 hours would cost about $255.50 at $0.35 per hour, before the host VM and other services. The equivalent accelerator line would be about $386.90 at $0.53 and $408.80 at $0.56. This is not a monthly quote: actual billing depends on the machine configuration, hours provisioned, discounts, region and any interruptions.
How managed inference APIs charge
Managed APIs remove most server operations, but they replace a GPU-hour meter with model-specific consumption meters. Always compare the same model family, input and output token counts, context length, region and service tier.
AWS Bedrock
Bedrock publishes prices for each supported model in input tokens and output tokens, normally expressed per 1 million tokens. It also lists charges for provisioned model units. Your bill therefore depends on the selected model and the ratio of generated to prompt tokens, not on a generic AWS GPU rate. A request-heavy application with short prompts can have a very different cost profile from a summarization pipeline that sends long documents and produces short answers.
Rank #2
- Original premium quality
- Item weight: 0.55 kg
- Size: Full-Height/Full-Length (FH/FL)
Microsoft Foundry
Microsoft Foundry documents token billing for model inputs and outputs and hourly hosting for some fine-tuned deployments. Its deployment choices include:
| Deployment type | Billing approach | Useful when |
|---|---|---|
| Global Standard | Pay per input and output token | General online inference where global data processing is acceptable |
| Global Batch | Pay per token with a documented 50% discount for eligible work | Large asynchronous jobs that can tolerate delay |
| Provisioned throughput | Reserved capacity units | Predictable demand, controlled latency and high utilization |
| Fine-tuned hosting | Hourly hosting for eligible deployments, plus usage terms where applicable | Serving a customized model rather than a catalog model |
The Global Batch discount applies to eligible Azure Foundry models and delay-tolerant asynchronous work. It is not a blanket 50% reduction for every Azure inference request. Confirm model eligibility, region and current terms before using it in a budget.
Google Cloud managed services
Google Cloud’s managed model catalog and serving products have their own token, throughput and capacity meters. They should be compared with Bedrock and Foundry only after selecting an equivalent model and service mode. If the model is not available in the required region or with the required quota, a lower nominal token price may not be usable.
Why a token-price comparison can give the wrong answer
A fair API comparison fixes the variables that most often move the result:
Rank #3
- NVIDIA Tesla T4 brings GPU Boost technology to boost performance of any application. Includes Error-Correcting-Codes (ECC) for protecting data reliability.
- PCI Express 5.0 host interface ensures dependable data transfer for maximum efficiency
- GDDR6 memory technology effectively enables data to be moved at various points in a CPU clock cycle to allow maximum productivity
- Plug-in Card form factor allows hassle-free and easy usage with increased efficiency
- Comes in 11.5" height for maximum productivity and easy carrying
- Model and version: Compare the same model capability and release, not similarly named but materially different models.
- Precision and context: Quantization and context length affect both quality and the amount of hardware required for a self-hosted model.
- Token mix: Record input and output tokens separately. Output tokens are often priced differently, and generation-heavy workloads can reverse a ranking.
- Traffic shape: Specify requests per second, daily volume, peak bursts and concurrency. Average volume alone hides the replicas needed for peaks.
- Latency target: A 200-millisecond target may require provisioned capacity that a best-effort batch job does not.
- Region and residency: A model or deployment may be global, regional or unavailable in the jurisdiction your data policy requires.
- Availability and failure handling: Retries, multi-zone replicas and cross-region failover add usage and infrastructure cost.
For a managed API, calculate input tokens multiplied by the provider’s input rate, plus output tokens multiplied by its output rate. Add any prompt-caching, batch, provisioned-capacity, fine-tuned-hosting or tool-use meters. Use the same request trace or representative dataset for every provider; otherwise a cheaper rate can simply reflect fewer tokens being counted.
How to calculate self-hosted total cost
GPU price is only one line in a self-hosted bill. Build a cost per hour and a cost per useful inference unit, such as 1,000 requests or 1 million output tokens.
- Choose the model and serving stack. Fix model version, quantization or precision, maximum context and batch size. Verify that the chosen GPU has enough memory.
- Measure capacity. Load-test at the target latency and concurrency. Record tokens per second, requests per second, queue time and error rate, including peak traffic rather than only a warm single request.
- Select the machine. Add the accelerator-hour charge to the VM or bare-metal host charge. Google Cloud explicitly notes that each GPU is charged in addition to the machine type.
- Add persistent and temporary storage. Include model disks, container images, logs, checkpoints and snapshots. Replicated or multi-zone storage can cost more than a single development disk.
- Add network and data transfer. Count ingress and egress, inter-zone traffic, private connectivity and traffic to retrieval or database systems.
- Add operations. Budget monitoring, log retention, orchestration, autoscaling, vulnerability scanning, backups, incident response and support.
- Model utilization. Divide useful serving time by provisioned time. A GPU that is paid for during overnight idle periods raises the effective cost per request.
- Price each purchasing mode. Produce separate on-demand, commitment and Spot scenarios. Do not blend them into one average that hides risk.
A useful formula is: (GPU + host + storage + network + operations + support + idle capacity) / useful inference output. For a cluster, multiply each component by the number of replicas needed for peak demand and availability, then subtract only credits or discounts that are actually guaranteed.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →On-demand, committed and Spot capacity
On-demand
On-demand capacity has the simplest accounting and the least commitment risk. It is a sensible baseline for a new workload, uncertain traffic or a service that cannot tolerate interruption. The T4 comparison above is on demand, so it is the cleanest starting point for comparing equivalent hardware.
Rank #4
Commitments and provisioned throughput
Google Cloud lists T4 commitment prices of $0.22 per hour for one year and $0.16 per hour for three years, compared with $0.35 on demand. Those lower rates require a commitment; they are not appropriate if demand may disappear or move to a different accelerator. Managed APIs use a similar economic idea through provisioned throughput: you pay for reserved capacity in exchange for predictable performance. The break-even point is the utilization at which the committed bill is lower than on-demand usage after including unused capacity.
Spot and interruptible capacity
Google Cloud describes typical Spot discounts of 60–91% for most machine types and GPUs. A separate August 2026 report puts observed Spot-to-on-demand examples for H100, A100 and V100 GPUs across the three hyperscalers at $1.95–$12.29 per GPU-hour. These figures vary by GPU, provider, region and time; they are not a guaranteed rate card.
Spot is a fit for resumable batch inference, evaluation, synthetic-data generation and other jobs that can checkpoint and retry. It is risky for a latency-sensitive endpoint unless you maintain on-demand fallback capacity. Include interruption recovery, queueing and duplicated work when comparing Spot with a stable instance.
Which option tends to be cheapest for each workload?
| Workload pattern | Usually worth evaluating first | Reason and caution |
|---|---|---|
| Low volume or unpredictable bursts | Managed pay-per-token API | You avoid paying for idle GPUs; check rate limits, cold starts and model availability. |
| Large asynchronous batch | Batch API or interruptible GPUs | Azure Global Batch documents a 50% discount for eligible models; compare equivalent batch programs elsewhere at the time of purchase. |
| Steady, high-throughput online traffic | Provisioned throughput or committed GPU capacity | Lower unit cost is possible when utilization is high enough to amortize the reservation. |
| Custom or fine-tuned model | Self-hosting or fine-tuned hosting | Catalog APIs may not support the model; include serving, updates and security work. |
| Strict regional residency | Regional managed deployment or self-hosting | A globally priced option may violate policy or be unavailable in the required region. |
| Interruption-tolerant evaluation | Spot GPUs | Low hourly rates can be offset by restarts and lost work without checkpointing. |
A practical comparison procedure
- Write the workload specification. Record model, version, precision, prompt-token distribution, completion-token distribution, requests per second, daily requests, peak concurrency, latency percentile, availability target and residency boundary.
- Create a provider matrix. For each cloud, record the exact SKU, region, deployment type, input rate, output rate, quota, minimum commitment and whether the model is available there.
- Run the managed-api calculation. Use the same token traces and add caching, batch, provisioned-capacity and fine-tuned-hosting meters. Keep input and output totals separate.
- Run the self-hosted calculation. Benchmark the model on each candidate GPU, then price enough replicas for peak demand and failure tolerance. Add non-GPU line items and idle time.
- Price three scenarios. Show on-demand, committed or provisioned, and Spot or interruptible capacity in separate columns. State what utilization and availability each assumes.
- Stress-test the assumptions. Recalculate with higher output-token volume, a lower utilization rate, a regional-only deployment and one failed replica. A choice that wins only under perfect utilization is not a robust production estimate.
- Record the date. Save region, SKU, model version, rate card, discount assumption and quota with the spreadsheet. Prices, catalogs and Spot behavior are volatile.
Is buying a GPU cheaper than renting one?
Ownership can make sense for a stable, heavily utilized workload, but a GPU sticker price is not an inference cost. A purchased NVIDIA T4, for example, still requires a compatible host, power, cooling, storage, networking, physical space, replacement planning and an operator. Divide the fully loaded capital and operating cost by the hours the system is actually serving useful work. Include depreciation, financing or opportunity cost, electricity, maintenance and the cost of keeping a spare if downtime matters.
Renting transfers hardware replacement and much of the facility burden to the cloud and lets you scale down. It can be more expensive per active hour but cheaper per completed request when demand is bursty. Buying becomes more attractive only after measured utilization, workload longevity and operational capability justify the fixed costs; the T4 hourly comparison does not by itself prove that ownership wins.
Common mistakes that make a “cheapest” answer unreliable
- Comparing a GPU-hour with a token price. These are different meters. Convert both to cost per identical workload.
- Using unlike models. A cheaper, smaller model may not meet quality or context requirements.
- Ignoring output tokens. Completion-heavy conversations can cost far more than prompt-only estimates suggest.
- Pricing only average traffic. Peak replicas, redundancy and queue limits determine the production bill.
- Assuming a discount is permanent. Commitments, batch discounts and Spot prices have conditions and can change.
- Leaving out idle time. A provisioned GPU accrues cost while waiting for requests.
- Ignoring data transfer and observability. Cross-zone traffic, logs and traces can become material at scale.
- Choosing a global service despite residency rules. A lower rate is irrelevant if the deployment cannot legally process the data.
- Failing to plan interruptions. Spot savings disappear when jobs restart repeatedly or require on-demand backup.
Decision checklist
- Have you selected the same model, version, precision and context limit for every option?
- Are input and output token volumes based on a real trace or representative test set?
- Did you specify region, residency, latency percentile, peak concurrency and availability?
- For GPUs, did you include the host machine, storage, network, monitoring, orchestration, support and idle capacity?
- Did you separate on-demand, commitment or provisioned, and Spot scenarios?
- Can the workload tolerate batching, queueing or interruptions?
- Have you verified current model availability, quotas, rate cards and discount eligibility on the date of purchase?
Verdict
Choose Google Cloud first for a T4-based self-hosting comparison: its published $0.35 on-demand rate is below AWS at $0.53 and Azure at $0.56, and its listed one- and three-year commitment rates are lower still. Do not turn that result into a blanket “Google is cheapest” conclusion. For bursty traffic, a managed token API can beat every always-on GPU by eliminating idle capacity. For predictable, high-volume traffic, committed GPUs or provisioned throughput can lower unit cost. For batch work, Azure Global Batch’s documented 50% discount may materially change the result when the model and workload qualify. The defensible answer is the provider and deployment that delivers your fixed model, token mix, region and latency target at the lowest fully loaded cost.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
