What I want to know is how are the able to tear down stone walls.

Just so weird that I have to protect my castle from wolves instead of the castle protecting me.