I'd keep the tree seeds, even if you don't use them to grow trees: much later in the game you can feed them to chickens. Seriously, they'll eat any seed and also barley (I haven't tried flax). Sure you'll run out of tree seeds eventually but it'll be nice to get a head start while building up farmed seeds.