The troll raid event only lasts two minutes or so. Get back to the trolls and have a look at the minimap. Stay within the red circle and either shoot the trolls with fire arrows or just kite them until the event timer runs out. After that they should move away and despawn. They should also be focused on you, so you can lead them away from your buildings.