As others have said, any raid can happen in any biome.
On a side note you can sandwich the dirt walls between stone walls to hide them. Then just rebuild wall sections as necessary. I usually don't worry about it. Trolls only attack your walls if you hide behind them. Go outside and fight em, or kite em till the event ends and lead em away.