Over investing early on in building a place is something you learn not to do after multiple playthroughs.

Anymore I just wait. Early raids are kind of a joke so you don't need defenses. Really only after the elder goes down.

Basically what I do is have a home at the first boss spawn with a fence around the garden area.

After I kill the first boss and work through the Black Forest I scout for a swamp with lots of crypts near a black forest... my current game I found an awesome spot.
Since I was going to have iron I set up my main base there for all the upgrades to stuff on a copper node in the black forest surrounded by swamp.
https://steamcommunity.com/sharedfiles/filedetails/?id=2791169888
Set some portals up on top of crypts to repair and get back from running iron out.

Set up base defenses and everything there before I beat the Elder.
Once I was set up for processing Iron there and base set up with portals I grabbed the key from the elder and was making iron that day. Running iron out, portal back.. processing it while I was digging it out and running more back.

It went super fast that way... few in game days and I am already in the mountains.

Now it depends on the mountain. If I have to go far I would make a new place to make silver and run iron over I may need but a great mountain is just across a channel from the swamp so I am just running the silver there. Off the mountain to a close shore and boat it back.

For me, the logistics of finding places and how I set up is the most interesting part of the game to me and keeps me playing multiple worlds since each one will present unique problems to solve due to the map RNG.