Maybe It's your dwelling design. I have two workbenches, one by the sea and another inside. The residence is a massive 3 story stone keep with a ditch lined with spikes circling it. Nothing ever spawns inside. If threats appear outside the perimeter I pick them off from the roof but they can't get in.