I haven't agro'd a dwarf yet and I have 3 extractors and a tower as a base. A 2-star seeker soldier will annihilate a tower full of dwarves and is more than ready to chase you to the next one. That said, I really feel like you should be able to trade for their goods if you so desire. Each camp is its own "faction" as far as agro goes. One camp will hate you forever if you steal from them but the next one won't know about it.