Breaking line of sight can help with this. Even with the spawn proofing, if they can see you they will come at you. They're drawn to light sources too. Even if they can't see you they might bang on your walls a little and will for sure bang on any standing torches left outside the walls and some of them will bang on your campfires. Blocking line of sight makes a difference and also works to hide your boat from them.

Even with trenches or earth wall you'll want to build back away from them because some mobs attacks will clip through.