Something that will help is using your pick to dig a moat around your base. Alternatively, you can use the hoe to build up a dirt wall around your base as well. In my case I do both. The enemies have a difficult time traversing steep grades. A little experimentation, and you will no doubt bolster your defenses. If you make your moats wide enough, you'll even pretty much cancel out the worry of ranged attackers. You can even build a wooden bridge to get across which you can break down when you need to afk.

At my main base, Dangerfjord, I used a hill in a small slice of meadow surrounded by black forests that also came down to the water. I cut into the hill for about half of my base and raised earth for the other half towards the water. Then, I built two long earth jetties out into the water so I could tuck a boat in 'mostly' safety. There is still enough of an earth bank in my harbor, where if enemies path out through the water, they still cannot get up. On the one side of my base it is a sheer cliff, and on the other, I built a labyrinth of raised earth that I put core wood poles across to slow trolls down. The labyrinth is exactly wide enough for a cart. To date, in almost 700 hours of playing, the only creatures that have made it into my base are oozes, and drakes. I've only seen those two particular raids a couple times. Most often I get Skeleton Surprise, Trolls, or Surtlings. No troll has ever seen the inside of my base, and the skelly surprise and surtling invasions are barely more than an inconvenience.

Another base I built I raised earth in a square shape as high as I could, leaving a gap in one face only wide enough for a single door, then built the base on top of that.

I cannot overstate how crucial the pick and the hoe are in this game. I cannot post screenshots here, but I'm sure you can find examples online. Good hunting!