Microsoft Introduces Maia 200 Chip to Advance AI Inference

Microsoft has announced the Maia 200, a purpose-built chip aimed at advancing artificial intelligence (AI) inference tasks. The new hardware features over 100 billion transistors and delivers more than 10 petaflops of computing performance at 4-bit precision, significantly improving upon the earlier Maia 100 model released in 2023.

AI inference—the stage where trained models generate outputs from incoming data—is a rapidly growing aspect of AI-related computing. As businesses expand their AI operations, inference workloads are contributing an increasing share of total computational costs. Microsoft positions the Maia 200 as an essential solution for running large-scale AI models more efficiently, with ambitions to reduce both operational disruptions and power consumption in data centers.

According to Microsoft, the Maia 200 can handle today's largest AI models and provides headroom for models that will require even greater computing resources in the future. "In practical terms, one Maia 200 node can effortlessly run today’s largest models, with plenty of headroom for even bigger models in the future," the company stated.

The launch comes amid a broader movement by major technology companies to develop in-house processors for AI. Historically, AI workloads have relied heavily on graphics processing units (GPUs) supplied by Nvidia, which has led to high costs and supply chain dependencies. To diversify their hardware portfolios and manage expenses, firms like Microsoft, Google, and Amazon are now rolling out their own AI accelerators. Google offers tensor processing units (TPUs) as cloud-based infrastructure, while Amazon recently debuted its third-generation Trainium chip.

Microsoft claims the Maia 200 outperforms both Amazon's Trainium and Google's latest TPUs in key performance benchmarks. The Maia 200 delivers triple the 4-bit (FP4) inference performance of Amazon’s Trainium 3 and exceeds Google’s seventh-generation TPU performance in 8-bit (FP8) computations.

The chip is already supporting internal AI developments at Microsoft, including the company's Superintelligence team and the Copilot chatbot. Microsoft has also begun to open up access via a software development kit (SDK), inviting developers, academics, and AI research labs to incorporate the Maia 200 into their own projects.

As the AI industry rapidly evolves, custom silicon like the Maia 200 is expected to play an increasingly important role in how organisations build, deploy, and scale advanced machine learning models.

Source: dataconomy.com