No sure if I am missing something here, but that's the idea behind PKI infrastructure: You can share your public keys without putting your identity at risk.

The "server knows who you are" or rather, I "tell the server who I am"

Interesting experiment though.