Edward Caban had been asked to quit after federal agents seized his phone as part of a criminal investigation, one of several involving members of the administration.