You can however it will be an uphill battle esp if solo.

We did this with a small island that had dark forest and plains biomes quite successfully and made it ourend game base. Only danger is event mobs so some replacement/repair is needed after events.

Existing and nearby spawning mobs will still wander over and attack things while you are building so you need to clear everything in sight as you go.

For me dark forest wasnt too bad, but for the plains I was only able to do this with a friend watching my back. For the swamp you may find that mobs like leeches and even some undead may spawn far out in water where you cannot build without mass terraforming.

Personally my friends and I found it easier to build a small portal shack on top of one of the crypts with enough room for a campfire and some chests. Just make sure its fully enclosed and mobs seem to ignore it, other than occasionally breaking whatever you place to help you jump up (WBench or small box works) We placed extra campfires around to supress mob spawns in the immediate area but these often needed replacing.

We disassembled our boat and stored it in the base until we had a few boxes of iron and rebuilt the boat to run it home.

P.S. Campfires are easier to place and achieve the same effect.