You will still get raided occasionally if you stay there too long at a time. But the workbenches will prevent random enemies from just spawning in there. Raids and enemies just generally spawning are separate.