What of people who were fertile but didn’t have children, through choice, medical reasons, death etc? Surely a person who is fertile but doesn’t have a child for any reason has the same net impact on the growth of population as an unfertile person (trans or not)?
Sure, but some causes of infertility are correlated and persistent.
If I'm exposed to poisons in the air or water, it's likely to happen to others as well - especially if it's at levels where it's not immediately noticed.
Many other causes of effective infertility are unlikely to impact entire groups of people across long periods of time.