Seekers can fly, no walls will protect you.

Your best bet is to get super lucky and find a section of "Tame" mistlands, that is, mistlands that only spawn hares. I have a spot on my server connected to mainland. Just a tiny sliver, but it's packed with roots, arable land for mushrooms, and bunnies spawn. I even found a small harbor with anglerfish. It was the holy grail of MIstlands real estate. The occasional goblin wanders in, but they're usually preoccupied chasing bunnies.