The game scales. The more you defeat the more night Mobs of higher teirs you ill get so build those walls & ditches as you go!
Make sure you have enough craft tables to cover the base area as that will cut down stuff spawning inside your walls as they usually show up just outside the radii of any crafting benches.