That's why this game desperately needs NPC guards imo. It kind of sucks that you need to resort to cheesing if you want to live anywhere other than Meadows.