Moats protect walls, walls protect whats behind them. Towers or other buildings have to be protected as well as the walls. Also, you should leave a little room between the iside of the moat and the wall. Stone is tougher, but clearly not invulnerable. And no, bosses will obliterate walls. I too as a solo player like to come up with creative approaches to minimize bosses, or "cheese" them as some would say, but beyond the swamps it gets hard to do. You could raise a terrain tower of sorts...nothing in the game can destroy that. I've wondered about it for the Yagluth fight, but I'm not sure how exactly to use them. Whatever the play style, flexibility and adaptability is key, imo.