As has been said, depends on your playstyle. Made my first main base before Elder. Afterwards I made a new and better one on a small remote island that's completely immune to raids. Stone building will unlock after Elder and you get some iron from the swamp, along with iron reinforced poles.
I now have 3 fortified based (meadows coast, island, plains coast) plus several smaller outpost for portal travel and access to certain biomes and resources.