Forget walls. Make earthen works higher than the mobs can jump over. It can't be destroyed as it is classed as terrain. Then put a wooden wall on the top with towers inside. The only weak point is the gate, but you can overcome that by using the AI against the mobs.