You can row the boat both forward and backward. The wind does not prevent you from pulling away from shore.

That's another way that the game is actually more generous than real-life. There's not way one person could row a boat the size of a longboat into storm-force winds, but Valheim lets you do it.