It's tricky to change up some parts of pip because it might break some existing users flows. It's really hard to test for this kind of infrastructure tool because it's so environment-dependent. The rewrite lets there be a clean slate. Meanwhile the target of being "compatible" with pip gives them a good idea of where to go, and they can focus a bit more on testing given the lessons learned.