Microsoft has introduced Maia 300, a custom AI accelerator intended for multi-gigawatt deployment from 2027. Announced on 12 October, the system is part of the company’s plan to make its own Maia hardware the primary inference platform for Microsoft products. Inference is the work of running a trained model to produce an answer, rather than training it in the first place.[1]
The notable unit is a complete rack: 96 accelerators, each with 288 GB of high-bandwidth memory, in a design rated at 167 kW. Microsoft describes a tightly integrated system in which data placement, chip-to-chip communication and cooling are designed together.[2]
A rack has to keep the chips fed
Maia 300 separates compute chiplets from memory-controller and input/output components. Its software-defined local dataflow lets software explicitly organise where data sits and when it moves. The accelerator’s eight HBM4 memory stacks provide a stated 18 TB/s of memory bandwidth; the rack combines local electrical connections with optical links between racks.[2]
Think of an answer-generation job as a series of handovers. One stage needs model weights, another needs an intermediate result, and a distributed stage may need information produced on another accelerator. Faster arithmetic helps only when those handovers arrive in time. The architectural argument is therefore about reducing waiting as well as increasing calculation speed.
What the rack specifications describe
| Specification | Published figure | Meaning |
|---|---|---|
| Accelerators in one rack | 96 | The planned system scale, not a count of racks already operating.Source 2 |
| Memory per accelerator | 288 GB | High-bandwidth memory local to each accelerator.Source 2 |
| Rack power | 167 kW | A stated design figure, not the electricity consumption of an individual answer.Source 2 |
Our calculation puts aggregate raw accelerator memory at 96 × 288 GB = 27,648 GB, or 27.648 decimal TB per rack. That is not a single model’s usable memory allowance: software, replication and the way a workload is partitioned affect what can actually be used.
Three efficiency claims, three different questions
Microsoft’s architecture account claims roughly 2.5 times Maia 200’s performance per watt and up to seven times its token throughput per rack. Separately, its broader announcement expects 40% more tokens per watt than leading GPUs. The first comparison concerns generations of Maia; the second concerns GPUs. These vendor comparisons need their own workload and system conditions to be meaningful.[2][3]
The arithmetic also deserves care. In a hypothetical like-for-like job, 40% more output per unit of power would mean the original output requires 1 ÷ 1.4, or about 71.4%, of the power. That is a 28.6% reduction, not 40%. Actual operating costs would also depend on utilisation, workload quality, cooling and the rest of the installation.
The existing deployment and the next one
Microsoft says Maia 200 already serves Copilot and Microsoft AI workloads, with GitHub Copilot and Foundry support planned by the end of 2026. Maia 300 is the next expansion, not evidence that the announced capacity is already serving customers.[1]
Designing the chip, rack and software together gives Microsoft several ways to improve a familiar service without changing its public interface. The same product can be served with a different memory arrangement, a different balance of computation and communication, or more work per unit of power. That is the practical significance of the announcement: a model’s capabilities may be visible in the chat window, but the economics increasingly depend on the system behind it.
Sources & further reading
- Microsoft to expand Azure Maia system to multi-gigawatt deployments Microsoft · 12 Oct 2026 · Accessed 14 Oct 2026
- Deep dive into Azure Maia 300 architecture Microsoft Azure · 12 Oct 2026 · Accessed 15 Oct 2026
- Infrastructure for the frontier era: Scaling intelligence through open industry collaboration Microsoft Azure · 13 Oct 2026 · Accessed 14 Oct 2026