Healthcare AI Firms Race to Cut Soaring GPU and Memory Costs

Artificial intelligence tools are quickly reshaping healthcare delivery, creating an intense need for capital and a surging appetite for advanced computing hardware. Across the technology sector, companies are buying high-performance graphics processing units (GPUs) and high-bandwidth memory (HBM) to run large-scale commercial language models and AI applications for their customers. Healthcare systems are beginning to face a similar reality. As AI-driven medical models and clinical applications become more common, forward-looking healthcare executives understand they will need substantial computing and memory infrastructure of their own. For some organizations, expanding relationships with established cloud providers may remain the right path. Others, however, are exploring sovereign, on-premise computing environments that they can control directly.

For years, relying on major cloud providers has been the standard approach, in part because those vendors typically offer broad packages of services, technical support and scalability. But as cloud and compute expenses continue to climb, the concept of a hospital-owned data center is gaining momentum. DataBank has outlined several reasons leaders are considering direct investment in computing infrastructure, including lower costs tied to observability and monitoring. Healthcare often depends on relatively consistent and predictable computing workloads, such as AI-enabled diagnostics and imaging analysis. Because these models can directly affect patient care, owning the compute environment can also give organizations stronger auditability and visibility, helping support safety, reliability and clinical effectiveness.

Another potential benefit is insulation from volatile pricing and supplier pressure. Much of today’s AI compute market is being shaped by the enormous daily usage of mainstream frontier AI models, which serve millions and possibly billions of users. As consumer demand rises, compute providers are confronting shortages, raising prices and racing to add more hardware capacity. For healthcare organizations, building or owning their own infrastructure could reduce exposure to those market swings and lessen dependence on public cloud platforms.

Healthcare leaders are also paying closer attention to the privacy and data-sovereignty implications of compute ownership. When an organization becomes vertically integrated—managing everything from applications to hardware across the service lifecycle—it gains far greater control over who holds its data and how that information is used. That approach can reduce cybersecurity exposure by limiting the number of external vendors with potential access through cloud or hardware systems. The HIPAA Journal recently reported that nearly 75,000 data breaches occurred in 2024 alone, continuing a year-over-year increase from 2023. As healthcare providers increasingly adopt AI applications, federated data systems and new tools embedded deeply into core information streams, cybersecurity incidents are likely to become an even larger concern over the next decade.

Still, sovereign compute is far from simple. Many healthcare organizations continue to outsource their infrastructure needs for a practical reason: specialized technology providers are built to manage these systems. While hospital-based data centers may offer clear advantages, they also present major challenges. Owning compute infrastructure requires ongoing maintenance, constant upgrades and dedicated technical experts. It can also be highly capital-intensive, demanding large upfront investments along with significant real estate and physical infrastructure. Time is another major factor. Even with experienced data center professionals and a more mature market, a typical hospital data center can take roughly two to five years to complete. For many health systems, partnering with an established technology or cloud provider that can deliver turnkey services and rapidly scale applications remains a logical choice, particularly because it removes much of the burden of maintenance costs and operational complexity.

As memory and chip prices continue to rise, healthcare organizations embracing advanced AI will have to make difficult decisions about cost, infrastructure and long-term control. Computing power is increasingly becoming as essential to 21st-century medicine as electricity or clean water, making hardware capacity one of the defining healthcare priorities of the decades ahead.