The next phase of AI infrastructure investment may depend as much on the changing economics and geography of computing as on the expansion of hyperscale data centers. Here's why that matters.
The Evolving AI Infrastructure Demand
The artificial intelligence infrastructure conversation has largely been dominated by the construction of increasingly enormous data centers, massive investments in GPU capacity, and the race among hyperscalers to develop the computing resources necessary to train increasingly sophisticated AI models.
That investment cycle is far from over. However, another development is beginning to reshape the industry's longer-term infrastructure requirements: the growing importance of AI inference.
Training establishes the capabilities of an AI model. Inference puts those capabilities to work, generating responses, analyzing information, executing tasks, and increasingly coordinating autonomous workflows.
As enterprise organizations move beyond experimentation toward production deployments, inference becomes an ongoing operational requirement rather than an occasional computational exercise.
This distinction has important implications for the economics and geography of AI computing. The infrastructure optimized for training enormous models is not necessarily the most economical or beneficial environment for every inference workload.
Enterprise applications, intelligent agents, industrial systems, and consumer services introduce different requirements involving response times, data proximity, network connectivity, energy efficiency, and operational control.
Consequently, the next phase of AI infrastructure development may involve something more significant than simply building additional hyperscale data center capacity. One size does not fit all requirements.
It may involve the emergence of a more heterogeneous computing ecosystem, combining centralized AI infrastructure with regional data centers, specialized inference facilities, and increasingly capable edge computing environments.
.jpg)
Inference Changes the Infrastructure Economics
The distinction between training and inference is becoming increasingly important to infrastructure investment decisions. This is a topic that demands the attention and consideration of the C-suite.
Training workloads generally benefit from highly concentrated computing resources, where large numbers of accelerators can operate together within tightly integrated environments. Inference, by comparison, encompasses a much broader range of operating conditions.
Some inference applications require enormous throughput and can be efficiently served from centralized facilities. Others depend on low latency, proximity to users or enterprise data, or the ability to operate within specific geographic and regulatory boundaries.
The economics are also different. Training represents a substantial investment in developing a model, while inference generates recurring computational demand as that model is used.
As AI adoption expands, the cumulative cost of running models, agents, and intelligent applications becomes an increasingly important consideration.
Research from JLL's 2026 Global Data Center Outlook anticipates that AI could represent approximately half of global data center workloads by 2030, with inference potentially overtaking training as the dominant AI requirement beginning in 2027.
McKinsey similarly identifies inference as a major driver of future data center development, particularly in metropolitan and near-metropolitan locations where network connectivity, energy efficiency, and proximity to users become important considerations.
These forecasts should not be interpreted as evidence that inference will automatically migrate away from hyperscale facilities. Rather, they indicate that the infrastructure market must accommodate a wider variety of computational requirements.
Beyond the Hyperscale Infrastructure Model
Several emerging developments illustrate how the infrastructure ecosystem could evolve.
Regional inference infrastructure. Data center operators are beginning to explore how geographically distributed facilities can provide access to AI computing resources closer to enterprise applications, proprietary data, and end users.
Equinix's September 2026 announcement of its Inference Exchange initiative, developed with NVIDIA and Together AI, is one example. The initiative is intended to connect distributed infrastructure with inference providers and enterprise customers across multiple metropolitan markets.
Smaller, modular data centers. The traditional assumption that meaningful AI computing capacity requires enormous facilities may become less applicable to certain inference workloads.
NVIDIA, Prologis, InfraPartners, and the Electric Power Research Institute have been exploring smaller data centers, ranging from approximately 5 MW to 20 MW, located near electrical substations. The objective is to make better use of available power capacity while enabling workloads to move between facilities as conditions change.
Telecommunications and edge infrastructure. Existing telecommunications facilities offer another potential deployment opportunity. Their geographic distribution, fiber connectivity, and established physical infrastructure could support smaller-scale inference environments.
This does not mean that every cell tower or telecommunications site is suitable for AI computing. Power availability, cooling, physical security, and connectivity remain important constraints. Nevertheless, the possibility of converting selected existing assets into distributed computing locations deserves attention.
Specialized inference architectures. Hardware and systems vendors are also exploring alternatives to conventional GPU-centric infrastructure. Purpose-built processors, optimized memory architectures, persistent KV caching, and more sophisticated networking can improve inference performance and resource utilization.
These developments could make smaller or more specialized computing environments economically viable for workloads that do not require the capabilities of the largest AI training clusters.
Together, these developments point toward an increasingly diverse infrastructure market.
The Geographic Importance of AI Computing
The location of computing resources has traditionally been determined by considerations such as real estate, electricity costs, network connectivity, and proximity to major cloud infrastructure.
AI inference introduces additional considerations.
An enterprise application supporting financial transactions, industrial automation, healthcare services, or intelligent customer interactions may have different requirements for latency, data residency, availability, and security. Processing information closer to where it originates or is consumed can offer operational advantages.
Power availability is another important consideration. Large data center projects increasingly encounter lengthy grid interconnection timelines and competition for electrical capacity. Smaller facilities located near available power infrastructure may provide an alternative deployment pathway, although they still require suitable cooling, redundancy, and network connectivity.
The result could be a more geographically distributed approach to AI computing, involving centralized AI factories, regional inference hubs, enterprise facilities, and edge deployments.
Importantly, this is unlikely to represent a wholesale migration away from hyperscale infrastructure. Centralized facilities will continue to offer substantial advantages in economies of scale, accelerator availability, and operational efficiency.
The more plausible outcome is a hybrid architecture in which computing resources are distributed according to workload requirements rather than concentrated according to a single infrastructure model.
There's another key consideration that has emerged, data center local regulation and site approvals. In the United States, finding a welcoming place to locate a new mega data center may become challenging.
The Counterargument: Hyperscale AI Still Matters
There are reasons to be cautious about declaring the arrival of a fully distributed AI infrastructure market.
Centralized infrastructure remains highly competitive for many inference applications, particularly where workloads can be aggregated and latency requirements are relatively flexible.
Regional and edge computing facilities introduce their own challenges, including higher operating costs per unit of capacity, fragmented deployment requirements, and the complexity of managing infrastructure across multiple locations.
Furthermore, the commercial viability of smaller facilities depends on securing sufficient and predictable workloads. Announced projects, prospective customers, and infrastructure partnerships should not be confused with demonstrated utilization or sustainable operating returns.
JLL's assessment is particularly instructive here. Its projected evolution of inference infrastructure begins with substantial centralization, followed by a gradual hybrid transition involving regional hubs and industry-specific clusters, and ultimately greater integration of edge intelligence.
This suggests that the transition will be evolutionary rather than immediate. It's yet to be determined.
The opportunity is not necessarily to replace hyperscale infrastructure, but to develop complementary computing environments that address requirements centralized facilities cannot always satisfy economically or operationally.
Market Outlook for Infrastructure Diversity
Several developments will help determine whether distributed inference becomes a substantial and commercially sustainable infrastructure market.
First, the economics of inference must continue to improve. Lower costs per completed task, more efficient accelerators, better memory utilization, and optimized networking will influence where organizations choose to deploy workloads.
Second, infrastructure providers must demonstrate that smaller and geographically distributed facilities can achieve attractive utilization and operating economics.
Third, telecommunications operators, colocation providers, utility partners, and regional infrastructure developers will need to establish commercially viable business models for delivering inference capacity.
Finally, enterprise adoption will be decisive. The expansion of production AI applications, particularly agentic systems that continuously interact with enterprise data and operational applications, could create sustained demand for geographically diverse computing resources.
The broader investment implications extend well beyond traditional data center operators. Semiconductor vendors, networking companies, storage providers, energy infrastructure developers, telecommunications operators, and specialized cloud providers may all participate in this evolving ecosystem.
A New Infrastructure Chapter for Applied-AI
The AI infrastructure investment cycle is entering a more complex phase.
The expansion of hyperscale computing remains essential to the development of increasingly capable AI models. However, the commercial value of those models will increasingly depend on the economics of putting them to work across business environments.
That changes the infrastructure equation.
The emerging opportunity is to determine where computing should occur, which architectures best serve particular workloads, how infrastructure resources can be utilized more efficiently, and how enterprises can maintain appropriate control over their data and AI operations.
For AI technology providers and enterprise decision-makers alike, understanding these changing requirements will become increasingly important. Anticipating data center location regulation is essential.
The next phase of AI infrastructure investment may be defined not just by the scale of computing, but by its purpose-built distribution and associated optimization.
Beyond hyperscale, a more heterogeneous and geographically distributed AI infrastructure ecosystem is beginning to take shape. Whether this develops into a substantial new market will depend on commercial adoption, demonstrated economics, and the ability of infrastructure providers to translate emerging architectures into dependable enterprise services.
For now, it represents an important development to watch as the tech industry moves from merely building AI capabilities toward delivering measurable business value from Applied-AI Initiatives.
Reach out to learn more about our Applied-AI Initiative objectives.