From my personal experience, there doesn't seem to be a minimum. I've been raided at a random workbench that had zero structures at it.

Ended up having multiple raids on my map, because I didn't stick around for that one.