The generative AI infrastructure stack is witnessing a remarkable surge in investment, as investors and enterprises see growing opportunities across the AI value chain.

Venture capital inflows into GenAI software and hardware infrastructure reached unprecedented levels this year, according to data from S&P Global Market Intelligence. With four months left until the year ends, the $13.2 billion recorded this year so far, is already nearly double the total of $6.8 billion recorded in 2023.

This increase in investment reflects a growing urgency among enterprises to integrate GenAI applications throughout their organizations. A recent survey by S&P Global Market Intelligence 451 Research reveals that a significant majority of over 700 respondents anticipate a rise in AI workload demands this year.

Nuts and bolts

While certain pockets of the market have garnered more attention than others, the GenAI infrastructure layer includes hardware components such as AI semiconductors, cloud offerings and datacenters, as well as software tools designed to enhance GenAI model performance.

Within the GenAI value chain, hardware infrastructure lies beneath the foundation model layer, while software infrastructure typically sits atop it. GenAI applications — such as OpenAI LLC's ChatGPT and Anthropic PBC's Claude AI — represent the final element in this value chain.

Many companies, particularly larger ones, operate across multiple areas within the GenAI value chain. For instance, Microsoft Corp. provides cloud infrastructure alongside GenAI applications like the ChatGPT-powered Copilot. Startups such as Cohere Inc. offer both data retrieval capabilities and specialized foundation models for enterprises. OpenAI and other mainstream foundation model providers also deliver applications in the form of chatbots.

Hardware

Hardware companies, including cloud infrastructure and semiconductors, are among the AI companies experiencing the highest increase in venture funding. While demand remains uncertain in the GenAI application space, semiconductors and cloud infrastructure attracted numerous customers, as evidenced by the stunning results of AI infrastructure giant NVIDIA Corp. Cloud infrastructure development is also capital intensive, necessitating larger funding rounds, including in the form of debt.

In 2024, specialized cloud providers such as Lambda Inc., Crusoe Energy Systems LLC, and Baseten Labs Inc. raised significant funds. Lambda's valuation surged to $1.8 billion in early 2024 from $35 million in 2021. The competition for a share of the rapidly expanding semiconductor market of graphics processing units, currently dominated by NVIDIA, has driven increased investment in semiconductors focused on AI workloads — particularly in inference, which is anticipated to become a larger market than training. While AI training teaches an AI model to perform a task, AI inference applies that training to new data.

Groq Inc., concentrating on inference chips, raised $600 million at a $2.8 billion valuation in 2024, nearly tripling its valuation since 2021. Both semiconductors and cloud infrastructure have already attracted more VC dollars than in the previous two years combined.

Software

While a great deal of attention has been on hardware, the software tooling space is gaining traction as stakeholders recognize that the performance of existing foundation models can be enhanced through methods like access to quality data, data retrieval techniques and fine-tuning, rather than merely increasing compute power.

Venture capitalist Marlon Nichols, managing partner at MaC Venture Capital, told Market Intelligence that MaC primarily invests in AI companies with proprietary data or unique data acquisition techniques, and avoids companies with no proprietary data or technology.

The non-deterministic nature of GenAI applications poses challenges for enterprises aiming to transition from experimentation to production.

"The simple truth is that working with GenAI models is still very difficult. There is no instruction manual and best practices are still undefined," venture capital firm Sapphire Ventures wrote in a recent blog post.

Consequently, a new wave of startups is emerging, promising to assist organizations in deploying AI more effectively. The startups offer tools that range from data preparation to GenAI development and deployment. Data management is one area gaining in importance among organizations.

"Enterprise data management has too often been framed in the context of externally imposed requirements such as regulatory compliance or reporting, rather than as an accelerant to business outcomes," said 451 Research analyst Paige Bartley. "Increasing interest in AI-related initiatives has renewed emphasis on the importance of data management practices."

Companies providing data management tools for GenAI applications raised over $1.7 billion in 2024, surpassing last year’s total, according to Market Intelligence data. Scale AI Inc. — which offers tools for data collection, curation and annotation for AI — raised $1 billion this year, doubling its valuation to $13.8 billion from $7.3 billion. Cohere, assisting companies in building AI models with their data, raised $500 million this year. The round sent its valuation surging to $5.5 billion from $2.2 billion.

Meanwhile, investments in GenAI security, model training, fine-tuning, deployment and GenAI app development have dipped below 2023 figures. Still, they could very well match or exceed the 2023 tally by year-end. Noteworthy funding rounds include Protect AI Inc., a platform offering tools to secure AI models, and Recogni Inc., which provides GenAI inference optimization.