Two things to think about.

First is workbench coverage.
Are you 100% certain there are no gaps where mobs could spawn inside? For example they could spawn inside when you get close and destroy things then despawn if you get further away... then you come back to find broken stuff and no mobs.

Second is raised ground.
When you manipulate the ground there can be issues.
Especially if on an incline but raised ground can suddenly drop out in spots then pop back up. I tend to work with terrain rather than modify it too much personally.
Are the broken things in places they are being supported by raised ground?

You can also try to run the game not as a server? Are you hosting it your machine or is it dedicated on a rental or something?