Here's my opinion that you totally asked for.

Ditch is good. Way too good. So I don't use them. So I'm mostly going to talk like ditches aren't a thing you can do, mostly.

Palisades. Only worth the investment if your base is built around geography that forces raids to walk through a bottleneck. Otherwise you would lose more time building hundreds of them in all directions than you would by skipping that step entirely and taking out raids by hand instead. They can pair with ditches, but simply catching them in a ditch is enough on its own to render enemies harmless.

Fences and stake walls are excellent. They are not there to protect you, they are there to protect your stuff. They are time buyers. And as long as a wall has more than 1hp and forces those stronger enemies to stop and hit it, it will buy you the time you need to get outside and round everything away from where you don't want them.
Stake walls are good for delaying pests up the the mountains. The weaker fences are an equally good, more cost effective way of slowing down enemies who will one-shot stake walls all the same.

Stone outer walls are overkill defence at an overkill price. They look good in an overcompensating kind of way. I wouldn't bother unless there were a bunch of hands collaborating. It would be more simple to double up or triple up on the wood options. That is more than enough time to gather up any troll or fuling raids that might ravage the place unattended. Not even the strongest of attacks can one shot its way into your base if you have more than one ring of barriers that are given sufficient spacing, unless you find a way to bring Moder or Yagluth home.

The best defence after ditches of course is YOU not being a fool who either fights inside their base or only pulls one enemy away from the group at a time. Freedom of movement will protect you better than any gear or built structure can and it takes zero maintenance.

tl;dr As long as you aren't dawdling at the start of the raid, stake walls and fences are all you need.