Higher-level mobs start attacking your base as you kill bosses.
Since you can move freely between the worlds, you can have one world where you progress by killing bosses and acquiring items to proceed in the next biome, and another world where you don't kill any bosses and just chill out building/exploring.
The only "raid" you'll have in that peaceful world is the boars/neck attacks triggered by the first boss, i.e. free food delivery to your doorstep.