The game doesn't even look at your surrounding biomes to determine what attacks you. You can be on a tiny island that is nothing but Meadows and get raided by goblins as long as there is space for them to spawn. Or you could set up in the plains and get raided by nothing but animals if you haven't defeated any bosses in that world yet. It's solely dependent upon the triggers, most of which are dependent upon which bosses have been killed.

You can actually use two worlds - kill the bosses in one and bring their trophies back to another. I did that just for fun after beating the game normally. I killed all the bosses a second time and stuck the trophies in a second world. In that world I have access to all the boss powers, but only get animal attacks at my base.