If he won't follow you, can't you just outrange him?

After you're done setting him on fire with arrows and stamping out whatever smouldering ashes are left, you can consider resummoning him to see if he winds up in the right spot next time.