Build workbenchs in these areas , there are still spawn point for raids even if you did a wall or a moat