The internal watchdog for the US Federal Housing Finance Agency is being removed from his role, four people familiar with the matter said.