> always seems to be some agile race to the bottom where we try to fix as little as possible as simply as possible.

Yeah, that's not agile. The entire point of agile is that you close the loop and the reason you break your work into smaller chunks is so you can deliver them faster in order to gauge the feedback of that chunk and better understand what should be delivered in the next chunk.

If you're not closing the loop, you're not doing agile, you're just doing waterfall with more bureaucracy.