It doesn't always work that way (which may be a bug). I've even had things spawn inside my base even though I have had full work bench coverage on my entire main base (birds and deer).
https://steamcommunity.com/sharedfiles/filedetails/?id=2860289081
https://steamcommunity.com/sharedfiles/filedetails/?id=2920176358

I've also had a troll raid spawn at my boat dock that only had a single work bench down. The boat was dismantled and inside a chest at the main base about 100 meters away. Since I didn't care about losing 20 wood. I just sprinted out of the area. No other raids can happen while one is all ready in progress and the raid timer doesn't count down if no players are in the raid area. The raid will despawn on its own after 3 game days if no players are in the area so you can't game it much but you can pretty much build in peace anywhere else for those 3 game days.

My experience is that there needs to be some fire source. If I have a teleport base with just the port plus the workbench I've never experienced a raid and likewise a farm base with just the workbench. But if I have a forge base then there is a fire source and I usually get the raid.

Originally posted by OctoberSky:

My experience is that there needs to be some fire source. If I have a teleport base with just the port plus the workbench I've never experienced a raid and likewise a farm base with just the workbench. But if I have a forge base then there is a fire source and I usually get the raid.

I had no fire source at my boat dock just a work bench and a boardwalk into the water.

From my personal experience, there doesn't seem to be a minimum. I've been raided at a random workbench that had zero structures at it.

Ended up having multiple raids on my map, because I didn't stick around for that one.