You learn pretty quick that strategy is constantly needed when playing this game. And for that you have to understand various mechanics like stamina and health. It's not easy to notice, but the raft has a ladder on the back side that lets you get back up. If you fell out and didn't see this then yeah your stamina would eventually run out from swimming, and subsequently your health. I have actually crossed pretty far just on a raft. It's possible to do it as long as you aren't cut off by a serpent.