I would argue that it takes a lot more spatial reasoning to balance over uneven terrain by manipulating many-jointed arms and legs then to wasd your way around a map that's mostly smooth.
And yet, there's plenty of research showing spatial memory improvements through video games. Funny how that works out - high level navigational skills can be improved by deliberate practice and tasks of adaptive difficulty, whereas engaging in ordinary tasks honed by eons of evolution don't much affect it?
That is to say, where is the link to math?