Why is Microsoft entrusting AI inference to its own chips?

Microsoft corporationCurrently developing in-houseAI chipBringing it closer to the core business. On October 12, Microsoft announced plans to scale the Azure Maia family to the teraflop level starting in 2027 and make it the primary inference system for Microsoft’s products and services. The next-generation Maia 300 will also begin production and deployment in 2027. While this is part of future plans, the Maia 200 is currently handling production workloads.Official Microsoft Announcement

What makes this arrangement noteworthy is that Microsoft aims to consolidate the demand for AI applications, chip design, and data center operations into a single ledger. As users continuously invoke models, inference costs will continue to accrue based on usage. Whoever can control the cost per valid service request will have greater leeway to determine product pricing, free usage quotas, and the scope of available features.

Power budgets are starting to affect how well AI products sell

Microsoft estimates that the Maia 300 can produce 40% more tokens per megawatt compared to leading GPU systems. Here, a “token” can be understood as the basic unit of content processing and generation by the model. This figure represents the manufacturer’s projected system efficiency; it should not be interpreted as a verified, general performance metric, nor should it be directly equated to a 40% reduction in total inference cost. The announcement did not provide complete comparison criteria sufficient for external parties to replicate the experiment.

The business implications of this metric are quite specific: assuming that the available power supply to a data center cannot be increased for the time being, higher power density offers the opportunity for existing facilities to handle more requests. Microsoft previously inAn Article on the Efficiency of AI InfrastructureIn this context, power consumption, memory, and system utilization have already been identified as key constraints. From this perspective, Maia is striving not only to secure pricing flexibility in the procurement phase but also to maximize service capacity within the constraints of power supply conditions.

However, turning capacity into revenue requires meeting product demand. The fact that machines generate more tokens does not necessarily mean users are willing to pay more. What businesses truly need is the accurate completion of search, analysis, or programming tasks, delivered within an acceptable timeframe. If low-precision computations or larger concurrent batches compromise task success rates, the theoretical throughput advantage may be offset by retries, waiting times, and manual verification.

Therefore, financial savings must also be viewed in context. Electricity costs are only part of the overall expenses; equipment procurement, data center renovations, R&D amortization, and maintenance personnel costs must also be factored in. While higher efficiency could be used to handle more free requests—potentially improving the customer experience—short-term profits may not necessarily increase accordingly. How the advantages of the chip will ultimately be allocated across price, functionality, and profit is a question that future product strategies will need to address.

Memory and Data Transfer Determine Whether Computing Power Can Be Utilized

The hardware trade-offs for the Maia 300 also revolve around inference. MicrosoftTechnical SpecificationsIt was disclosed that the compute die uses TSMC’s N2P process, while memory control and I/O functions are located on separate dies; each accelerator is equipped with 288 GB of HBM4 memory, with a peak bandwidth of 18 TB/s. To be precise, while the compute die uses the N2P process, this does not mean the entire package can be described as being manufactured using the same process.

Long contexts cause reasoning systems to maintain more intermediate states. Continuous follow-up questions from users and repeated tool invocations by agents also prolong the time these states are occupied. In such cases, even if the computing units are very powerful, they may still experience delays if data is not delivered in a timely manner. Memory capacity determines how many tasks can be handled simultaneously, while bandwidth and data layout affect the speed at which computing units can access data.

This explains why the competition in the chip industry cannot be judged solely by peak computing power. For those operating AI services, high computing power that goes unused is no different from idle servers—both incur depreciation and maintenance costs. Only by optimizing tasks for the hardware and minimizing data detours can hardware specifications be translated into marketable service capabilities. Memory expansion itself also entails procurement and power costs; ultimately, it comes down to the return on investment of the entire system.

Microsoft's advantage lies in the fact that it already has real demand.

According to this announcement, Maia 200 is already being used for production workloads such as Microsoft Copilot and supports the Microsoft MAI team; GitHub Copilot and Microsoft Foundry are scheduled to follow suit by the end of the year. The former has already been deployed, while the latter remains on the roadmap. For Microsoft, its own applications can provide continuous workload feedback, helping the hardware team determine which computing patterns are most worth optimizing.

This is one of the economic factors driving large platforms to develop their own chips. Chip development requires upfront investment, while specific models and user behavior are still evolving. Without a sufficiently stable volume of business, the cost savings from custom designs may not cover the costs of R&D and software integration. By owning application gateways, model services, and cloud infrastructure, Microsoft at least has the opportunity to internally offset these costs.

However, having more options doesn't mean that general-purpose GPUs have lost their value. Microsoft also announced in July thatExpanding Our Collaboration with AMD in AI and High-Performance Computing. Given its continued emphasis on a heterogeneous platform strategy, a more reasonable conclusion is that Microsoft aims to allocate hardware based on workload, deploying its in-house systems where they can best demonstrate their advantages. Interpreting the Maia project as a complete replacement for NVIDIA goes beyond the evidence currently available.

Software compatibility and operational efficiency must both meet the requirements.

Microsoft’s technical documentation also describes designs for managing data locations, scheduling tasks across racks, and utilizing spare resources. The single-rack configuration of the Maia 300 accommodates 96 accelerators with a power budget of 167 kilowatts. This raises a practical issue: once chips enter the data center, power supply, thermal management, scheduling, and fault recovery all impact overall efficiency.

Inference requests do not arrive evenly throughout the day, and model sizes and task lengths vary. If a suitable combination of resources cannot be found for a task, some devices may remain idle while others are stuck in long queues. Therefore, the value of compilers, operator optimization, and scheduling systems lies not only in making a single test run faster, but also in helping the entire fleet of machines maintain a consistent, controllable output.

The in-house development approach also places more responsibility on Microsoft. When new models are launched, software support must keep pace; when hardware issues arise, diagnostics and recovery must not compromise the customer experience. The most persuasive evidence going forward should include clear metrics on sustained throughput, availability, and total cost under specific model and latency requirements—rather than simply comparing the best figures from different scenarios.

What Chinese companies should really learn is how to crunch the numbers on this.

For domestic cloud providers and model development companies, Maia offers a perspective on assessing the value of in-house systems: first, identify long-standing, sufficiently concentrated business workloads, and then determine what the hardware should be optimized for. The decision on whether to develop in-house chips should be made in conjunction with the model update cycle, the capabilities of the software team, and the actual deployment volume. The number of effective requests over which R&D costs are spread often provides a more accurate commercial benchmark than the specifications of a single chip.

For Chinese companies using AI—including those expanding overseas and purchasing cloud services abroad—procurement issues are even more immediate. When comparing quotes, response latency, context length, output quality, and peak concurrency conditions should be aligned; when comparing costs, factors such as retries, caching, manual review, and migration and maintenance should be factored in. The lowest price per million tokens does not necessarily correspond to the lowest business processing cost.

Whether the Maia 300 can deliver on its promises will depend on actual deployment and operational data in 2027. What can be confirmed for now is that Microsoft is attempting to improve the unit economics of its AI services by gaining more control at the infrastructure level. This will bring industry competition closer to a fundamental question: within a given power and budget constraints, how much work that customers are willing to pay for can actually be consistently delivered?

Information as of October 13, 2026 (Beijing Time). Source:Microsoft Maia Extension Announcement,Maia 300 Architecture Overview,Microsoft Infrastructure Efficiency Articles,Announcement of the Partnership Between Microsoft and AMD. Performance and deployment information are provided by the vendor; business judgments are based on the analysis in this article.