A single NVIDIA H100 costs between $27,000 and $40,000 to buy, a B200 runs $30,000 to $50,000, and a fully built GB200 NVL72 rack can reach $3 million or more. AI GPU pricing spans three orders of magnitude depending on the chip, the form factor, and whether you buy a card or a complete system. For any team budgeting AI infrastructure, that capital number is only the start, which is why hardware spend belongs inside a FinOps practice from day one.

This guide breaks AI GPU pricing into four parts:

• Per-chip prices for the GPUs everyone trains on

• Full system and DGX prices once networking and CPUs are added

• Price per performance, so the sticker number is not read in isolation

• Buy versus rent, the decision that actually moves the budget

Per-Chip AI GPU Prices

Purchase prices for the data-center cards, per GPU:

NVIDIA card prices, including an 8-GPU H200 configuration quoted near $315,000, come from a current price guide. The GB200 superchip figure is from reported Blackwell pricing, and the H200 and MI300X per-GPU prices and memory specs from a spec comparison.

A few buying notes:

• Memory tier drives the price: The jump from A100 40GB to 80GB adds roughly $5,000 for the same chip.

• Secondhand cards are increasingly available: Used H100 and A100 supply has grown as teams upgrade to newer Blackwell parts, pulling resale prices below new.

• The H100 vs A100 decision is the most common one: If a model fits in A100 memory, the H100 premium is wasted, a trade-off covered in H100 vs A100.

Full System and DGX Prices

A bare GPU is not a usable machine. Once CPUs, networking, storage, and chassis are added, the real number climbs:

System prices come from a current server pricing guide.

What inflates a system above the raw card cost:

• High-speed interconnect (NVLink, InfiniBand) for multi-GPU training

• Grace CPUs and host memory on superchip configurations

• Storage and networking sized to keep the GPUs fed

• Power and cooling for racks that can draw over 100kW

The point that matters for budgeting: choosing the card is about more than its specs, it is about cost per finished run, which is the core of picking the right GPU for training.

Price Per Performance

The sticker price means little without throughput. Normalized to compute:

Price-per-performance figures come from a price-performance analysis.

How to use these numbers:

• A higher sticker price can be cheaper per result. A B200 costs more than an H100 but can finish the same work for less.

• Match the chip to the bottleneck. Memory-bound inference favors high-HBM parts like the MI300X; compute-bound training favors Blackwell.

• Generational jumps reset the math. Newer silicon usually wins on cost per TFLOP even at a higher headline price.

Buy vs Rent

The biggest budget decision is whether to own the hardware at all:

• Buy when utilization is high and steady. Owning an H100 pays off only if it runs most of the time. At low utilization, a $30,000 card sits idle while depreciating.

• Rent when demand is spiky or experimental. Cloud H100 capacity rents from roughly $2 to $8 per GPU per hour, with spot rates near $1.43 (rental rates), so short jobs almost never justify a purchase.

• Blend for most teams. Own a baseline fleet for steady inference, rent for training bursts and new experiments.

Whichever path you take, the spend has to be tracked, because owned GPUs hide waste just as easily as rented ones:

• Owned hardware buries cost in idle racks and underused capacity, often inside Kubernetes clusters where Kubernetes cost management surfaces the waste.

• Rented GPUs drift on the invoice as usage scales.

• Either way, every GPU hour needs an owner. Amnic lets you allocate cost back to teams and models so no spend goes untagged, the foundation of FinOps for AI.

Does AI GPU Pricing Vary by Region?

Yes. The same chip carries a different price depending on where you rent or buy it, driven by energy costs, supply, and trade rules.

Cloud rental rates by region, per GPU per hour:

North America stays the cheapest and most stable, while European rates run higher at the top of the range on account of energy costs and thinner spot supply (regional pricing data). Within a single cloud, US regions are usually the lowest, with other regions adding a surcharge.

Trade rules also shape availability and price:

• Allied nations such as the US, UK, Canada, Japan and Germany face no procurement restrictions and get the best pricing and supply.

• H100, H200 and Blackwell parts are barred from a set of restricted countries, which fragments supply and lifts prices in markets that rely on limited channels.

• Stripped-down variants such as the H20 serve restricted markets in place of the full chips, so the headline price does not map to the same performance everywhere.

The practical takeaway:

• Price the workload in the region you will actually run it, not the cheapest list rate you can find.

• Factor energy cost into owned hardware, since a region with high power prices raises the real cost of every GPU-hour.

Real Cost Examples

The sticker prices only matter once you run the math on a real workload. Four worked examples, using the rates above:

These figures exclude power, cooling, and depreciation on owned hardware.

How to Keep AI GPU Spend Under Control

A short checklist regardless of chip or buying model:

• Normalize to cost per useful output, a finished training run or per million tokens, not the purchase price or hourly rate

• Tag every GPU to a team, model, or workload from the start

• Track utilization, since idle high-end cards are the single largest source of waste

• Compare buy and rent on total cost of ownership, including power, cooling, and depreciation

• Alert on drift so a cost change surfaces while a job is still running

Conclusion

AI GPU pricing runs from a $10,000 A100 to a near $3 million Blackwell rack, and the right number depends on the chip, the system around it, the work it does, and whether you own it. The prices will keep moving as supply loosens and new silicon ships. What stays constant is the need to see GPU spend clearly, allocate it accurately, and catch cost anomalies before they compound.

FAQs

How much does an NVIDIA H100 cost?

A new H100 costs $27,000 to $40,000 per GPU. Used and secondhand cards trade below that as supply eases and teams upgrade to newer Blackwell hardware.

How much does a DGX server cost?

A DGX H100 server runs about $290,000, a DGX B200 about $515,000, and a full GB200 NVL72 rack reaches roughly $3,000,000.

What is the cheapest AI GPU to buy?

Among data-center cards, the A100 40GB is the lowest at about $10,000 to $12,000. The AMD MI300X is also competitively priced at roughly $10,000 to $15,000 with more memory than the H200.

Is the B200 worth more than the H100?

Often yes. A B200 costs more per chip but delivers roughly $8 to $12 per FP16 TFLOP against the H100's $16 to $20, so it can finish the same work for less.

Should I buy or rent AI GPUs?

Buy when utilization is high and steady, since an owned H100 only pays off if it runs most of the time. Rent for spiky or experimental work, where cloud H100s start near $2 per GPU per hour.

How does the AMD MI300X compare on price?

The MI300X is priced around $10,000 to $15,000 per GPU and offers more memory than the H200, 192GB versus 141GB, making it a strong value for memory-bound inference.

Does AI GPU pricing vary by region?

Yes. North American cloud rates are the lowest, often $1.99 to $4.12 per H100 GPU-hour, while European rates reach $6.39 on higher energy costs. Trade controls also bar H100, H200 and Blackwell chips from some countries, raising prices where supply is limited.