A great many of the illegal pot farmers damage public (and private) lands, screw with the water supply, pollute rivers, and leave trash behind. Not to mention growing country has a higher rate of violent crime than even LA County. That's where the asshole part comes in.
