Originally posted by gutlor:

So, what am I supposed to do? Run off till they completely destroy everything then come back and rebuild it with a moat? Start a new game and concentrate on all defenses first?

You get 100% of your materials back so you won't have to chop trees all over again. The wood and whatever else you have will be on the ground waiting for you to pick it up again. So, yeah, rebuild after you run out the timer. It take a few days for items on the ground to despawn, and they never despawn when in range of a workbench. You lost time, but not materials.