What is the memory bandwidth to the main CPU cores (not the "neural engine")? Is it really 120GB/s like they say in the spec sheet[0]? That's 20GB/s faster than the top dual-channel DDR5 desktops, which makes me think there might be some fine print I'm missing.
It uses HBM. It’s faster because it’s millimeters from the CPU/GPU and has a wide bus. If you think 120GB/s is high you should look up the Max/Ultra chips’ specs.
Memory bandwidth for M series goes up to 800 Gb/s for Ultra (in e.g. Mac Studio). That, combined with the ability to get it loaded with a lot of RAM, is what makes it so desirable for LLMs.
[0] https://www.apple.com/au/macbook-air/specs/