Note that unless you're camping out within site of him he won't move at all. Everything basically hibernates in place when their part of the map is inactive, i.e. you're not within visual range of them.