The countdown timer is reset every time the controller level is downgraded. From the article Territory Control:
A Controller cannot be damaged or destroyed. However, in case there are no spawns of the owner player in the room left, a countdown timer is started and the Controller starts losing 1 level every 50,000 game ticks. As soon as its level reaches 0, it becomes neutral, and another player can reclaim it.