After hearth and home update, there is a constant spawn arount player, so you (or your buildings) will be attacked every few minutes.
Killing bosses also changes night spawns, and after Eikthyr, you will be attacked by waves of greydwarfs (with shamans).

The best "workaround" is to play the game without killing bosses (you can get everything up to silver without killing any) and tame some wolves/boars to protect your buildings.