its just a random chance you'll be attacked.

Few things, A. Make buildings out of stone. Troll invades are no joke and enemies tend to attack lights and crafting tables they don't tend to care about anything else.

B. If you leave during the raid on your base (which only happens at your spawn point) it will RESPAWN when you come back. Don't leave the area during the fight or the things you're fighting just respawn and attack you again.