Settling for multiple smaller patches could be more safe, and never planting all of your veggies at once. The bigger area you want for your crops, the more vulnerable it will be. I have enough space for just about 12 crops behind my initial portal tower and no enemy has ever approached it from the side of the crops thanks to ridges and water acting as natural obstacles.