Try building your trench around an empty area to start. Raids need to have three base constructions, like workbench, fire, bed, or other interactables, to trigger. You can either remove everything but the bare walls, or dig around an empty field to prevent raids while you're putting up defenses.