I feel the same way.
I suspect that people are having a knee-jerk reaction because it sounds like more (and boring) work. Just like if someone asked you to write comments to every line of code. Hence the argument "sounds good on paper" which seems to always come up. To me, TDD doesn't sound good on paper at all. It just turned out to be more fun (to my surprise) because it is much easier for me to get into flow.