There's a disconnect here in that your argument presumes that we are talking about groups of people which organically and unanimously decided to install a CoC as a guideline or explanation on how exactly they are going to exercise their right to free association. However, the scenery that opponents of the CoC-associated culture typically see, and consider the modal scenario to be opposed, is that CoC proponents demand that groups of people that do not initially include them (or at least that they are a fairly marginal minority in) institute CoCs. Quite often, this also includes an implicit or explicit demand to install an outsider belonging to the "CoC proponent" group in a position that has wide-ranging powers to interpret the CoC. (After all, someone not properly trained might commit a rookie mistake such as thinking that in the story this thread is about, the enforcers were actually the ones who violated the CoC!) Consider the now-famous example of the person who first wanted to impose a CoC on Ruby canvassing/thinking out loud about removing Matz from his BDFL position.

You could argue that, if someone tells me to install a CoC or else (they will complain to my employer, and spread the message that I am a bad person), they are just exercising their right to free association and/or helping my employer and anyone who may see that messaging exercise theirs, but I think that this is very close to being a free association counterpart of the "free speech" defense of blatant slanderous misinformation, or people using a public list of businesses operated by members of a minority to inform their non-patronage.