If you have a workbench and a fire next to your bed, and you probably do, it will also get raided. Having a raid every 3-4 days is just unlucky. It shouldn't always be like that.

You should have access to a pickaxe now, so dig a ditch with steep sides all around your base. Make sure that every inch of the inside area is covered by a workbench build radius. (Other things can stop spawns too, but the workbench lets you see the area it covers.) Mobs, including trolls can't climb vertical earth walls. If you align your digging with the compass, you can get smooth straight edges instead of jagged ones.

Leave room around the inside edges because the trolls will be doing AoE damage later and can throw big rocks. Make the inside area bigger than you think you need because you'll be adding a garden and more work spaces later. It usually takes me about 2-3 in-game days to complete a ditch, but it's well worth the effort.