Groq raised $640 million on August 5, 2024, at a $2.8 billion valuation to expand its specialized AI-inference hardware and GroqCloud service. The company was not trying to replace Nvidia across training, gaming, scientific computing, and every form of accelerated computing. Its narrower bet was that fast, predictable inference—the process of running trained models for users—could support a major business alongside Nvidia’s broader GPU platform.
That bet made Groq a serious Nvidia challenger in a specific segment, but the story later became more complicated. Groq raised additional capital, shifted greater emphasis toward its inference cloud, and entered a non-exclusive inference-technology licensing agreement with Nvidia in December 2025. As of August 2026, GroqCloud remained a separate business.
What Groq’s $640 million round funded
Groq announced the Series D financing on August 5, 2024. Funds and accounts managed by BlackRock Private Equity Partners led the round. Named participants included Neuberger Berman, Type One Ventures, Cisco Investments, Global Brain’s KDDI Open Innovation Fund III, Samsung Catalyst Fund, and existing investors.
The financing valued Groq at $2.8 billion. The company said it would use the money to expand GroqCloud, including the planned deployment of more than 100,000 additional Language Processing Units, or LPUs. That distinction matters: the round was not simply research funding for a chip designer. It was also capital for data-center capacity, cloud operations, software, customer acquisition, and the utilization risk that comes with building an inference service.
#1 Best Overall
- PLEASE NOTE: Exporting an NVIDIA RTX Pro 6000 GPU outside the US requires strict adherence to the U.S. Export Administration Regulations (EAR) and issuance of an export license from the Bureau of Industry and Security (BIS). Compliance and Know Your Customer (KYC) screening may be required as a condition of order acceptance. [NVIDIA Blackwell Streaming Multiprocessor] The new SM features increased processing throughput, and new neural shaders that integrate neural networks inside of programmable shaders | DLSS 4: Multi Frame Generation ensures ultra-smooth frame pacing for lifelike simulations.
- [Double-Flow-Through Design] The RTX PRO 6000 Blackwell features a double-flow-through cooling design, optimizing efficiency and airflow to sustain peak performance under 600W power loads. | [5th Gen Tensor Cores] Deliver up to 3X the performance of the previous generation and support for FP4 precision for faster AI model processing times with reduced memory usage, enabling local fine-tuning of LLMs and generative AI | [4th Gen Ray Tracing Cores] Double the ray-triangle intersection rate of the previous generation to create photoreal, physically accurate scenes and immersive 3D designs with RTX Mega Geometry, which enables up to 100X more ray-traced triangles.
- [PCIe Gen 5] Support for PCIe Gen 5 provides double the bandwidth of PCIe Gen 4, improving data-transfer speeds from CPU memory and unlocking faster performance for data-intensive tasks like AI, data science, and 3D modeling. | [GDDR7 Memory] With 96 GB of GPU memory and 1.8 TB ps bandwidth, it can tackle massive 3D and AI projects, fine-tune AI models locally, explore large-scale VR environments, and drive larger multi-app workflows.
- [DisplayPort 2.1] Achieve unparalleled visual clarity and performance, driving high resolution displays at up to 8K at 240 Hz and 16K at 60 Hz. Increased bandwidth enables seamless multi-monitor setups while HDR and higher color depth support ensures superior color accuracy for precision work, such as video editing, 3D design, and live broadcasting.
- [Universal MIG] Divide a single RTX PRO 6000 Blackwell into multiple isolated instances, each with dedicated resources, allowing for concurrent execution of multiple workloads, optimized GPU utilization, and secure isolation of different applications or users. [WARRANTY] 3 YR Manufacturer's Warranty. Bulk OEM Packaging. Retail Packaging is NOT included.
Groq had previously raised approximately $300 million in a major April 2021 financing at a valuation of roughly $1 billion, according to contemporaneous reporting. Groq’s funding announcement describes the Series D and its planned capacity expansion.
Why Groq focused on AI inference
AI infrastructure has two related but different jobs:
- Training uses large amounts of computation to build or adapt a model. It typically requires massive parallel processing, substantial memory bandwidth, and extensive software support.
- Inference runs an already-trained model to produce an answer, prediction, transcription, or other result for an application or user.
Nvidia’s GPUs serve both markets, but Groq concentrated on inference. Once an AI application has millions of users, the cost and responsiveness of repeatedly serving model requests can matter as much as the initial training run. A provider that can generate responses quickly and predictably may be attractive for voice assistants, conversational applications, search, agents, and other interactive systems.
Groq’s commercial pitch was therefore about lower latency, high token-generation rates, and potentially better economics for selected models and workloads. It was not a claim that a specialized inference processor could replace Nvidia in large-scale training or general-purpose accelerated computing.
What is Groq’s LPU?
Groq calls its specialized processor a Language Processing Unit. It is designed specifically for neural-network inference rather than serving as a general-purpose GPU that can accommodate a very broad range of workloads.
Rank #2
- NVIDIA Ampere Architecture-based CUDA Cores - Double-speed processing for single-precision floating point (FP32) operations and improved power efficiency provide significant performance improvements for graphics and simulation workflows, such as complex 3D computer-aided design (CAD) and computer-aided engineering (CAE), on the desktop.
- Second-Generation RT Cores - With up to 2X the throughput over the previous generation and the ability to concurrently run ray tracing with either shading or denoising capabilities, second-generation RT Cores deliver massive speedups for workloads like photorealistic rendering of movie content, architectural design evaluations, and virtual prototyping of product designs. This technology also speeds up the rendering of ray-traced motion blur for faster results with greater visual accuracy.
- Third-Generation Tensor Cores - New Tensor Float 32 (TF32) precision provides up to 5X the training throughput over the previous generation to accelerate AI and data science model training without requiring any code changes. Hardware support for structural sparsity doubles the throughput for inferencing. Tensor Cores also bring AI to graphics with capabilities like DLSS, AI denoising, and enhanced editing for select applications.
- Third-Generation NVIDIA NVLink - Increased GPU-to-GPU interconnect bandwidth provides a single scalable memory to accelerate graphics and compute workloads and tackle larger datasets.
- 48 Gigabytes (GB) of GPU Memory - Ultra-fast GDDR6 memory, scalable up to 96 GB with NVLink, gives data scientists, engineers, and creative professionals the large memory necessary to work with massive datasets and workloads like data science and simulation.
At a high level, Groq controls both the LPU architecture and much of the software stack used to compile and run supported models. The design emphasizes predictable execution and high throughput. That can make performance easier to plan for than a system whose results vary substantially with workload, batching, or contention.
But an LPU is not simply a faster GPU, and “faster than Nvidia” is not a complete technical claim. Results depend on the model, compiler and software support, context length, batch size, concurrency, networking, queueing, and the latency metric being measured. A processor optimized for supported language models may be less flexible when a customer needs an unusual operator, a new model architecture, custom kernels, or broad CUDA compatibility.
Groq’s own LPU overview emphasizes inference performance. Its latency guidance also warns that console measurements represent server-side latency; network time between an application and the service is additional.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsGroq hardware and GroqCloud
Groq’s business has two connected parts:
- Groq hardware: specialized inference accelerators intended for data-center deployment by customers or infrastructure partners.
- GroqCloud: a hosted API that lets developers run supported models without buying, installing, and operating Groq hardware themselves.
The cloud strategy gave Groq a way to turn a chip advantage into a usable product. Developers could call an API instead of managing accelerators, drivers, networking, and capacity planning. Groq could also capture more of the economics and customer relationship than it might through hardware sales alone.
That approach introduces its own obligations. Groq must finance and operate enough capacity, support production customers, maintain API and model compatibility, manage geographic availability, and keep utilization high enough to justify the infrastructure. The $640 million raise was consequently a bet on an AI-inference cloud, not merely on the technical merits of a processor.
Rank #3
- NVIDIA Volta GV100 Architecture — 4,608 CUDA Cores, 640 1st-Gen Tensor Cores delivering 14 TFLOPS FP32 and 112 TFLOPS deep learning performance for AI training, inference, HPC, and scientific computing workloads
- 32GB HBM2 ECC Memory — 900 GB/s Bandwidth — High-bandwidth memory on a 4096-bit bus with ECC error correction provides the memory capacity and throughput required for the largest AI models, simulations, and datasets
- PCIe 3.0 x16 Interface — 250W TDP — Standard PCIe Gen3 connectivity with passive cooling designed for enterprise rack server deployment in HPE ProLiant, Dell PowerEdge, and Supermicro platforms with adequate chassis airflow
- NVLink — Scale to 96GB Unified Memory — Connect two V100 GPUs via NVLink at 300 GB/s bi-directional bandwidth to scale GPU memory from 32GB to 96GB for larger AI training and HPC workloads
- Multi-Precision Computing — Supports FP64 (7 TFLOPS), FP32 (14 TFLOPS), FP16 (112 TFLOPS) and INT8 precision modes for flexible deployment across training, inference, and scientific simulation workloads
Why Nvidia was the comparison point
Nvidia’s advantage extends well beyond its chips. It has a large installed base, CUDA and related developer tools, mature libraries, networking, systems expertise, and established relationships with major cloud providers. Customers can use Nvidia infrastructure for training, fine-tuning, inference, simulation, analytics, and other accelerated workloads.
Groq’s potential advantage was more focused:
- Specialized performance for supported inference workloads.
- A possible latency or cost benefit for particular models.
- An alternative source of inference capacity for customers seeking less dependence on Nvidia GPUs.
- An API-based service that reduced hardware-management work.
That is why “challenge Nvidia” was directionally accurate but too broad if interpreted as an attempt to displace Nvidia across the entire AI-compute market. Groq’s credible competitive target was Nvidia’s inference position, especially where response speed and predictable throughput were commercially important.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallWhere Groq could make sense
GroqCloud may be compelling when an application:
- Needs low response latency or fast streaming output.
- Serves high volumes of requests using relatively stable model architectures.
- Uses models already supported by GroqCloud.
- Benefits from an API rather than operating accelerator infrastructure.
- Needs another inference supplier alongside Nvidia-based systems.
Groq’s model documentation has displayed examples such as Llama 3.1 8B Instant at $0.05 per million input tokens and $0.08 per million output tokens, and Llama 3.3 70B Versatile at $0.59 per million input tokens and $0.79 per million output tokens. It has also listed approximate speeds of 560 tokens per second for the 8B model and 280 tokens per second for the 70B model. These are live operational figures, not permanent specifications; prices, model IDs, speeds, context windows, and limits can change. Check the current model documentation before making a purchasing decision.
For production use, the relevant comparison is not one headline tokens-per-second number. Buyers should measure:
- Time to first token.
- End-to-end response latency.
- Sustained output tokens per second.
- Input and output cost.
- Cost per completed task rather than cost per token alone.
- Concurrency, batch size, and context length.
- Queue latency and availability.
- Model quality, quantization, and tool-calling behavior.
- Network distance and streaming performance.
- Rate limits and committed enterprise capacity.
Where Groq faced obstacles
Specialization versus flexibility
A purpose-built accelerator can perform very well on targeted workloads while requiring more compiler and engineering work for new architectures. Customers should ask how quickly new models are supported, whether custom models can be brought to the platform, which operations are unavailable, and how much portability they would have outside GroqCloud.
Rank #4
- Standard Memory: 40 GB
- Host Interface: PCI Express 4.0
- Cooler Type: Passive Cooler
- Product Type: Graphics Card
Existing Nvidia investment
Organizations that already own Nvidia clusters or have deeply integrated CUDA software face switching costs. Nvidia may remain preferable for customers that train and serve models on the same platform, rely on custom kernels, need unusual operators, or want one standard spanning training, fine-tuning, inference, simulation, and analytics.
Cloud and procurement constraints
GroqCloud reduces hardware friction but creates dependence on Groq’s capacity, regions, pricing, data-handling terms, API behavior, and model-retirement policies. Enterprises may also favor AWS, Microsoft Azure, or Google Cloud because of existing contracts, private networking, identity controls, compliance programs, and procurement processes. A faster specialized service does not automatically overcome those practical barriers.
Benchmarks are not production economics
Vendor speed figures may use a particular model, prompt length, output length, batch size, and server configuration. They may exclude network latency and peak-time queueing. A higher generation rate is valuable only if it produces an acceptable answer at the required quality, reliability, and total cost.
Capacity and rate limits
Groq documents organization-level limits for requests, tokens, and audio. Exceeding a limit can produce an HTTP 429 response. The default on_demand service is convenient but can experience queue latency during peaks. flex processing is intended for higher throughput but may return over-capacity errors. The enterprise-only performance tier provides provisioned throughput and documents 99.9% availability and a 99% latency guarantee subject to the enterprise agreement. See Groq’s service-tier, flex-processing, performance-tier, and rate-limit documentation for current terms.
What happened after the 2024 financing?
| Date | Development | Why it matters |
|---|---|---|
| August 5, 2024 | Groq announces a $640 million Series D at a $2.8 billion valuation. | Capital is aimed at expanding GroqCloud and adding more than 100,000 planned LPUs. |
| September 17, 2025 | Groq announces a $750 million financing at a $6.9 billion post-money valuation. | The company’s funding and valuation expand as demand for inference grows. |
| December 2025 | Groq and Nvidia enter a non-exclusive inference-technology licensing agreement. | The competitive relationship becomes partly collaborative rather than a simple head-to-head contest. |
| June 22, 2026 | Groq announces $650 million in growth capital. | The company continues emphasizing the scale-up of its inference cloud. |
Later reporting described Nvidia hiring much of Groq’s senior technical team while Groq continued as an independent company focused on GroqCloud. The arrangement should not be casually described as Nvidia acquiring Groq. The company-backed description is a non-exclusive licensing agreement; reported personnel and investor arrangements are separate matters. Groq’s newsroom and the 2026 TechCrunch report provide the later-status context.
How to evaluate GroqCloud today
GroqCloud is a reasonable candidate for teams prioritizing latency, API simplicity, and the economics of supported models. A hyperscaler may be a better fit when governance, private networking, regional deployment, procurement, and broad model choice dominate. Nvidia infrastructure is generally the stronger choice when the organization needs training, fine-tuning, CUDA portability, custom kernels, and control over the complete stack.
Groq should not be treated as universally fastest or cheapest. The correct choice depends on the model, prompt and output distribution, concurrency, geography, service tier, SLA, and the cost of the surrounding application. Alternatives include OpenAI, Anthropic, Google Vertex AI, Amazon Bedrock, Microsoft Azure AI Foundry, Nvidia NIM, and Cerebras Inference, but comparisons should use the buyer’s actual workload rather than general marketing claims.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
