Sam Altman stated on the Cheeky Pint podcast that placing RAM in personal computers is inefficient compared to hosting it within data centers. He argued that while some users desire local inference, the assumption that open-source models will make data centre infrastructure obsolete is incorrect. Altman described this distributed approach as terrible from a global allocation perspective because high-bandwidth memory remains a scarce resource. His comments address the current shortage of computer memory that has affected pricing and availability for hardware manufacturers.
The statement highlights a fundamental constraint in building large language models that require massive amounts of fast memory to function effectively. This issue does not imply changes to customer products or API behaviour, but rather reflects the physical limits of computing hardware. OpenAI continues to rely on specialised cloud infrastructure to train and run its systems rather than distributing the load to individual machines.
* High-bandwidth memory shortages currently impact the supply chain for consumer electronics.
* Altman noted that local inference is not a viable replacement for centralised data centres.
* The interview took place during a discussion on AI’s impact on mathematics.
