There is a very rare bug where certain building pieces will repeatedly break for no known reason. I had a small wooden shack just outside a swamp where one specific wall would break if I left the area and returned. It had some kind of error when loading in apparently.

I could fix it and sleep in it all night, then go out and explore and it would be broken every time I returned. That one shack was the only building in all of my runs that did that. It wasn't stability and it wasn't from mobs attacking it.

That said, it's a very rare bug, so check the obvious things already mentioned first.