Chill away! The game difficulty progresses with your progress in relation to mobs and bosses. You may get raided by Boars and Necks, but unless you kill a boss, a Troll (arguably harder than first boss) or a Surtling, the world will not progress in difficulty.