Stone walls are durable enough to usually weather troll attacks. Just make sure you put them at a bit of a distance from anything breakable.