The president merely has control of the heads of departments. They don't have control over the rank and file. It is notoriously hard to get rid of non-elected government officials...