I share your concern. Airlines seem to be anticipating this. There was a recent publicized incident of American Airlines removing a woman from a flight for playing audio over her phone speakers. United has similar policies. As I understand it, both are saying they will ban passengers over it.

United recently started announcing at the beginning of flights that using your phone with sound on the speaker is prohibited, as a new official policy.

Talking on your phone has always been prohibited while in flight. Having high speed internet on a plane is not new.