You should be able to get just about anywhere at row speed in a half hour. You don't need the wind to go places. It's just slower. Were you just sitting there with your sails still up and cursing your fate instead of rowing (which is speed 1 on the boat, rowing icon appears, sails rolled up) You're never really "dead in the water."