> The possibility of having a valid email address where the second-level domain component can't actually be registered as a domain, to the best of my understanding, eg "bar!.com", is interesting.

It is! Conceivably, it could be used to implement a new kind of domain-less mail system, e.g. foo@$megamail or something.