Was there actually a barrier at exascale? I mean, was this like the sound barrier in flight where there is some discontinuity that requires some qualitatively different approaches? Or is the headline just a fancy way of saying, "look how big/fast it is!"
One thing is having a bunch of computers. Another thing is to get the working efficiently on the same problem.
While I'm not sure exascale was something like the sound barrier, I do know a lot of hard work has been done to be able to efficiently utilize such large clusters.
Especially the interconnects and network topology can make a huge difference in efficiency[1] and Cray's Slingshot interconnect[2], used in Aurora[3], is an important part of that[4].
Diagonalization of working set and scaling-up and -out coordination. Some programs (algorithms) just have >= O(n) time and space, temporal-dependent "map" or "reduce" steps that require enormous amounts of "shared" storage and/or "IPC".
I specifically said I wasn't sure it's a barrier. My point was that you can't scale up without hard work. That is, you can't scale just by buying more of the same hardware.
It isn't comparable to the sound barrier, but it was still a challenge.
It took significantly longer than it should have if it was just business as usual: "At a supercomputing conference in 2009, Computerworld projected exascale implementation by 2018." [1]
We got the first true exascale system with Frontier in 2022.
Part of the problem was the power consumption and having a purely CPU based system online for an exascale job. From slide 12 from[2]: "Aggressive design is at 70 MW" and "HW failure every 35 minutes".
Yeah. Back in the 90s, "terascale" was the same kind of milestone buzzword that was being thrown around all the time.
Because of that, I felt a bit of nostalgia when I first saw consumer-accessible GPUs hitting the 1 TFLOP performance level, which now I suppose qualifies as a cheap iGPU.
I have no problem with “exascale” or whatever, it’s just not a barrier, there was nothing in particular to overcome to get there, it’s just a signpost along the normal path forwards.
Unlike the sound barrier in flight, it was for a while thought impossible to fly faster than sound and there are indeed models of subsonic flight that have an infinity in them at the speed of sound. It indeed took new models and radically different designs to fly faster than sound.
Making this computer do a certain number of calculations didn’t involve any such problem to overcome.
