I would say it's more likely that you missed them OR accidentally planted them and forgot about it until you were back in the area. The saplings don't grow unless you plant them. All the trees are just cosmetic resources you can harvest.