If Google were to integrate generative AI into every search, its electricity use would rise by roughly 29bn kilowatt-hours per year, according to a paper published in energy journal Joule last year. That’s about the same amount the entire population of Iceland uses annually.
The stat encapsulates one of the biggest challenges facing our future alongside AI — training and running these models requires substantial energy. The amount needed is only going to go up.
Companies are, however, starting to cotton on to the opportunity to tackle AI’s energy problem. One of them, University College London (UCL) spinout Oriole Networks, has just raised a £10m seed round. Investors include XTX Ventures, the venture arm of algorithmic trading firm XTX; Clean Growth Fund; Dorilton Ventures and the UCL Technology Fund.
Building better networks
Training AI models relies on graphic processing units (GPUs) such as those produced by Nvidia. Training tools like large language models (LLMs) — the tech that powers chatbots like ChatGPT — means clustering thousands of GPUs together. XTX has a supercomputer made up of 20k GPUs, for example.
Today, GPUs are usually connected through ethernet cables (a higher-performance version of the cable that comes out of a WiFi box).
Over the last 10 years, GPUs have advanced significantly, but the networks that connect them into clusters have not. “The network has become the bottleneck,” says James Regan, CEO of Oriole — which was founded in April last year.
Ethernet networks can slow down the compute power of the cluster when they hit a certain amount of information passing through them. They also consume a significant amount of energy — roughly 20% of the total consumption of the overall AI cluster (the rest is from computer processing itself).
Oriole’s solution
Oriole’s solution comes from the work of Professor George Zervas at UCL, who’s been working on improving computer networks since his PhD 20 years ago. Zervas cofounded Oriole and is now its CTO.
Zervas has created a way to connect GPUs using light beams running through optical fibres. The tech can increase the speed at which packets of information can travel between GPUs by up to 100 times, Regan says.
Optical networks use far less energy than traditional ethernet networks, continues Regan. At the lab scale, Oriole says its tech has cut the network's energy consumption to 2-3% of a traditional system.
Other companies are working on optical networks: Google, for example, has a project called Mission Apollo. “Google’s announcement is great,” says Regan – though he adds Oriole’s tech can be faster and cheaper because it replaces more network elements with optical systems.
Commercialising the product
Oriole has licensed the tech from UCL Business (the university's tech transfer arm). The company’s IP covers the physical architecture of the optical system, plus the machine learning that makes it possible.
Oriole plans to outsource the manufacturing of the hardware to companies which are already producing network infrastructure. It won’t sell entire supercomputers, it plans to just sell the networking system: so when a customer pieces together its computing power, it can select Oriole’s tech in place of ethernet cables.
Once product development is complete, it’ll start selling its tech to customers. Regan estimates that will take a couple of years. “Now we've actually got to go build it,” he says. “All the simulations and the prototyping and testing in the lab have shown that this works.”
