Do you actually have empirical data demonstrating that scientists aren't concerned about AI, or that "most" AI safety doesn't involve scientists, or is this just your gut feeling?
Edit: for instance, here's a computer science professor and neurologist who published a book about all of the serious dangers of AI, and calling for funding safety research:
Furthermore, AI safety programs are hiring AI researchers and computer scientists to actually do the work, so are you claiming that these people don't sincerely believe in the work they're doing?
An argument that depends on equating priests and scientists is an argument that fails.