Why not just build a safe base up in the tall, indestructible trees of the swamp?...

It is completely possible (on vanilla game) to build in those trees and even be able to place campfire or hearth as well as smelter and coal kiln, if you do it right.

Here is one I built which is clearly much higher than any land mob can reach

https://steamcommunity.com/sharedfiles/filedetails/?id=2661983674

Hearth placed in the tree house in the swamps

https://steamcommunity.com/sharedfiles/filedetails/?id=2643987894

Smelter and coal kiln placed up in the tree house in the swamps

https://steamcommunity.com/sharedfiles/filedetails/?id=2643988771

Those big trees in the swamp can't be destroyed and they act as "ground" point for your building... Every piece touching the tree is blue (structural integrity), so it's quite easy to build up out of reach of most mobs.