Don't invest in a crappy raft unless it's for going over a river - any Sea Serpent will eat you otherwise. As for swimming you need to plan a bit for that + it's worth leveling it up when are initially exploring. Food with high stamina is good.

Otherwise you need to rely on portals until you gain a certain boss power which helps in sailing.