As mentioned in another thread it's mainly down to confirmation bias.

You remember the annoying experiences much clearer and more vividly than the non annoying experiences, which gives you a biased impression of their relative proportions.

So its not just you, it feels like that for everybody. But it isn't actually true.

If you want to prove it to yourself you'd have to do the actual research, ie pick a good length straight line journey back and forth between 2 places and time exactly how long in total the wind is against or behind you, do it several times to get an accurate average.

In actual fact I'd predict the wind would be favourable roughly 75% of the time in accordance with the 75% coverage between all your points of sail. (between 1:30 & 10:30 on the clockface)

Though to be honest I've not actually done this, I've just felt the rage myself so many times and was lucky enough to learn precisely how pseudo random number generators work at a machine code level. There might be some algorithm that 'tailors' the wind direction to 'tease' the player but this seems like an illogical design choice. The major problem PRNG has is that it's essentially too random, whereas real randoms allow for some exceptionally apparently ordered sequences, like rolling 1 on a dice in excess of 10 times in a row etc. PRNGs really struggle to produce sequences like this. Don't laugh I have had this happen to me more than once.