xAI Colossus Supercomputer Cluster

The story

In mid-2024 Elon Musk's xAI converted a vacant 785,000-square-foot Electrolux appliance factory in Memphis into the world's dense AI training cluster in 122 days. Named Colossus, the cluster connected 100,000 liquid-cooled NVIDIA H100 GPUs across an Ethernet fabric without multiple tiers of oversubscription. To meet the cluster's massive power demands before utility substation upgrades finished, xAI deployed 14 natural gas turbines alongside a 50 MW Tesla Megapack battery installation. Colossus trained the Grok-3 foundation model and began doubling toward 200,000 GPUs.

Notable

• Assembled 100,000 liquid-cooled NVIDIA H100 GPUs on a single interconnected RDMA fabric in just 122 days.
• Housed in a converted 785,000-square-foot former Electrolux appliance manufacturing plant in Southwest Memphis.
• Powered by 150 MW of utility capacity from MLGW plus 14 mobile natural gas turbines and Tesla Megapack batteries.
• Doubling capacity toward 200,000 GPUs including next-generation NVIDIA Blackwell architectures.

Expansions

• 2024: site acquired at former Electrolux plant; 100,000 liquid-cooled H100 GPUs brought live in 122 days
• 2024: 14 mobile natural gas turbines and Tesla Megapacks deployed for supplemental power
• 2025: expansion underway to double cluster size to 200,000 accelerators including NVIDIA Blackwell