I had a similar adventure finding my Elder. Long long long sail until I hit an island that I knew he was on the other side of. I chose to go counter-clockwise which as it turned out was ten times longer than the other way. Not knowing the shapes of the landmasses makes for some serious adventures, and you can't sail too close to plains biomes or the skeeters will fly out and get you.

Best advice is bring lots of good food, so you don't get squished by a troll when you step ashore like I did. (I'd forgotten to eat and was malnourished.......... otherwise I could've killed him even though he suckered me).

Like you, I wandered both into the swamp and a plains biome before I was ready and wow did I skeedaddle. I was a long way from my portal near where I'd killed the Elder and snuffing it in such a location would've been an unmitigated disaster. This game will bend you over a log and pound you mercilessly for being careless. Never forget that.

Also never forget to have a spare set of armor ready for the day that you screw up big time and have to out, build another boat and go get your stuff back. Been there.

Crossing the swamp when you're not ready is perhaps not the best advice. Scout it CAREFULLY for sure but know how to make a bonfire (for abominations), and stock up on potions. Pull a mob to you with a bow to see how you handle it.

You're not getting any iron until you kill the Elder.