In all my survival games I tend to keep to the water.
I found a nice peninsula that I made an island by digging a moat around it.
When a raid happens go out and fight, no fun in hiding in the base. I use an old copper I mined out as a kind of arena with a safe place of unreachable high ground in the middle.
I put sharp stakes in choke points, you can even line the whole moat with them so as mobs run through it they take damage and die.