It's all random. While building my tavern, I had a raid almost every day, later none for almost an ingame week.

Where I think I'm cursed is that my raids almost invariably happen at nightfall. I HATE that! :D