Raids will happen. It is very bad luck in your case. But that's why it is so important to have a portal nearby. It is also important to clear out any adds in the vicinity of the fight and your portal BEFORE the fight.

Though you should not have gotten a bat event. That tells me you've built up an entire base there (or what Valheim considers one). There should be no need for that, if you have a portal not too far away.

Trust me, I had pretty bad luck with almost continuous Troll raids recently. They've taught me to hate Trolls again... :P

Good luck next time! You can do it!