If your home is not on the shore close enough that you can do a short hop to get the iron from boat to smelter, you may want to take the smelter to the iron. That is annoying too, though: You need a lot of forge upgrades to make or upgrade some items.

I find it really easiest to build on the shore, portal to the swamp, boat back home. It does not get any better than this without cheating.

Not exactly cheating, but boating does not use stamina. You can ride home loaded down with tons of iron in your inventory. You won't be able to shoot much this way, so your route needs to be safe and so on.

You don't have to leave your base behind. But if you decide you want to do that, maybe put the next one within a short distance of the ocean with good water access to all the biomes.