Groq secures $640M to supercharge AI inference with next-gen LPUs

Groq, a leader in AI inference technology, has raised $640 million in a Series D funding round, signaling a major shift in the artificial intelligence infrastructure landscape. The investment values the company at $2.8 billion and was led by BlackRock Private Equity Partners, with participation from Neuberger Berman, Type One Ventures, and strategic investors such as Cisco, KDDI, and Samsung Catalyst Fund.

The Mountain View-based company will use the funds to rapidly scale its capacity and accelerate the development of its next-generation Language Processing Unit (LPU). This move addresses the AI industry’s urgent need for faster inference capabilities as it shifts focus from training to deployment.

Stuart Pann, Groq’s recently appointed Chief Operating Officer, emphasized the company’s readiness to meet this demand in an interview with VentureBeat. “We already have the orders in place with our suppliers, we are developing a robust rack manufacturing approach with ODM partners, and we have procured the necessary data center space and power to build out our cloud,” Pann said.

The Silicon Valley speedster: Groq’s race to the top

Groq plans to deploy over 108,000 LPUs by the end of Q1 2025, positioning itself to become the largest AI inference compute capacity provider outside of major tech giants. This expansion supports Groq’s swelling developer base, which now exceeds 356,000 users building on the company’s GroqCloud platform.

The company’s tokens-as-a-service (TaaS) offering has garnered attention for its speed and cost-effectiveness. Pann told VentureBeat, “Groq offers Tokens-as-a-Service on its GroqCloud and is not only the fastest, but the most affordable as measured by independent benchmarks from Artificial Analysis. We call this inference economics.”

Chips and dips: Navigating the semiconductor storm

Groq’s supply chain strategy sets it apart in an industry plagued by chip shortages. “The LPU is a fundamentally different architecture that doesn’t rely on components that have extended lead times,” Pann said. “It does not use HBM memory or CoWos packaging and is built on a GlobalFoundries 14 nm process that is cost effective, mature, and built in the United States.”

This focus on domestic manufacturing aligns with growing concerns about supply chain security in the tech sector. It also positions Groq favorably amid increasing government scrutiny of AI technologies and their origins.

The rapid adoption of Groq’s technology has led to diverse applications. Pann highlighted several use cases, including “patient coordination and care, dynamic pricing by analyzing market demand and adjusting prices in real-time, and processing an entire genome in real-time to get up-to-date gene drug guidelines using LLMs.”