Its usually after a few minutes. If your base is well defended with a moat/trench that mobs can't scale, you can typically just wait them out.