Just stick to the shore, nothing to worry about. Had all the same during my first sea trials.
Avoid deep water and be on the lookout for serpents; aside from that, the farther you go the weather will change again.
If you can afford it build a Karve, better than a simple raft.

Rafts are dangerous contraptions. Ill use one early if my island is too small, to go from a shore I am on to a shore I can see. Otherwise I wait on the karve which is my go-to ship, the longship is too big for my tastes unless hauling a lot of ore.

The sea is a bit dangerous. Storms and waves won't hurt you but the serpents and stray biome critters will.
Honestly, the serpents always swim off if I shoot them 3-4 times. They are 'mostly harmless' unless in the raft.
What gets you out at sea, more often than not, is simply the stray biome critters.
the swamp's leeches will swim way out and chew on your boat. Undead archers will go under and shoot you from the depths. Trolls can wade out a fair distance and smash at you. Small humanoids and animals will swim at you but usually cannot catch you or do anything. Dethsquitos will dive bomb you and they can sink a boat really fast, you need to shoot them down if you are unlucky enough to get one to chase you.

All that boils down to your original thought -- storms, night/dark, fog etc are dangerous just because you cannot see the shore type ahead of you and may stimulate some nasties into sinking your boat. Its best to explore on clear days when the sun is up. If you are caught in a storm, you can stop where you are if you are lost or confused. The minimap is your friend here, you can 'see' what you can't see using it but the range of discovery is a bit limited.

Enjoy the sea, and be very wary of boating into the unknown, and you will be just fine otherwise. If you didn't know, 1 click forward is row against the wind, one click back is reverse and inverts your turn controls like a car.

The weather and waves get more rough when you enter the ocean biome. So stay closer to the coast in order to not leave the "land" biome.
Although if you want to encounter the leviathan or sea serpents you need to go to the ocean biome but don't do it in a raft.

You can do fishing from the land, no need to sail unless you want to hunt the special „fish“ (sea serpent).

Keep that raft in shallow waters--it be dangerous in the ocean. For fishing, I just go up and down the coast and toss a line out when I see some fish, or sometimes just angle for deeper water and let it sit, usually a pike or tuna will come by to snag it. Keep that stamina high and you'll have a stack of fish in no time.

Thank you all for the greatly appreciated comments.