KOVE
– where you can ’ t amortise it across the infrastructure – is on the edge .”
He adds : “ Data centre people have been struggling to reduce costs for some time . Commodification has been one answer , but even that cannot work anymore .
“ Now , people have needs that exceed the limits of what a data centre can physically hold , and the only way to do it is either to be more clever on the software stack or to control the memory and power surfaces .”
Crucially , establishing control over the memory surface removes the limit of how much memory can be housed by a given host server .
“ Once you control the memory surface you remove these limits ,” says Overton . “ So instead of figuring out how to cram computations into a physical platform you can focus on creating a broader and more flexible physical platform that can be deployed on-demand to support arriving needs that cannot otherwise be addressed .”
And this is key , because today ’ s largelanguage models require enormous amounts of memory . This means that – without solutions like Kove : SDM™ – data centres have little choice but to resort to parallelisation , whereby multiple servers are networked in a cluster . Although this can work well , the power demands are colossal and it can lead to resource stranding . The world ’ s largest supercomputer , Aurora – based at the Argonne National Laboratory – is reported to consume up to 60 megawatts of power at its peak : the equivalent of more than
84 June 2024