Originally posted by Maelstrom:

Mobs don't spawn within 20 yards of any base structure. Walls / floors / plants don't count. See the wiki page https://valheim.fandom.com/wiki/Player_base

Easiest way is use workbenches since their range is marked with a dotted line.

My fort was close by but I guess it wasn't close enough. Still sucks to have to manage spawns in such a fasion. Especially the plains required farms.

Thanks for the tips.