Interview with Robert Linsdell, EkkoSense
DigitalBridge CEO Marc Ganzi recently said that the new specialised AI chips – GPUs – that NVIDIA, AMD and Intel are producing consume two to three times the power of prior generations – as much as a toaster. He forecast that data centre power will increasingly be dominated by these AI workloads and reckoning that 80% of future data centre power will be consumed by AI workloads over the next 15 years.
Data centre operators are between a rock and a hard place as a result. They need to maximise efficiency and maintain SLAs without risking resilience. To do this they need to provide the visibility required to improve efficiency to either reduce energy, release capacity or to identify design challenges which require attention.
AI will cause data centres to be redesigned in Australia and New Zealand so they can handle high heat loads, even while demand intensifies. Industry veteran Robert Linsdell has just been appointed GM ANZ and APAC of AI-powered data centre optimisation company EkkoSense. He believes the operators who will successfully navigate to AI operations will balance their cooling and IT loads, and use their power and space more efficiently.
“Having visibility of what the white space is doing and using tools such as monitoring ensures SLAs are maintained and infrastructure is tuned to to deliver where cooling is needed whether it’s air or liquid or hybrid,” he said. “A 10% efficiency gain is power that can be used for something else.”
Linsdell told W.Media that while so-called AI-factory data centres are emerging, most local data centre operators are incorporating AI into existing operations. “Companies…are making the AI piece as part of their existing structures so placing a pod at the end of their data centres, or some are buried in the middle of a large hall to concentrate the cooling,” he said. “There are a few immersion pods around but they are all experimental except when used for research.”
Longer term he believes Australia and New Zealand will see adjunct AI-only buildings [next to cloud data centres] but they will need to be where the power is located. “Some companies will consider small pods. AI tends to be few racks, high-density cooling 50 to 100kW a rack and lots of power, he said. “And DLC or high density rear door cooling seems to be the way forward.”
Linsdell said the emergence of AI training data centres and inference data centres is leading bigger cloud customers to work closely with their hyperscaler of choice – essentially working with one they know. A GPT-3-scale LLM – around 1300 MWh with 175 billion parameters – requires the equivalent per training session of one hour of power from one nuclear power plant – approximately 1000 MWh. It also requires a large GPU cluster to operate and the cost of tuning and inference, using generic models, for specific industries is enormous.
So while smaller data centres may not see much more than the occasional one or two racks for AI, he said he has already seen some interesting smaller new entrants being far more active with AI. For example, given the breadth of AI, not every player will be focused on LLMs with billions of parameters – plenty of companies are exploring smaller models that may be used by a specific company in a closed environment. For example, NTT’s tsuzumi AI model has a lightweight version which is designed to perform high-speed inference on a single GPU, while the ultra-lightweight version can do so on a CPU.
Linsdell said AI will challenge the sector’s climate goals. “Because of the big power loads – it is possible to have very tight closed loop systems so the PUEs look good but the power drain is huge,” he said. “It is likely the benefits downstream will be needed to justify the upstream climate cost, in my opinion.”
[Author: Simon Dux]
Comments