Like Warchild said, they do have a large aggro range.

Do what I do, I kill the Growths then set up a workbench at the edge of the tar pit. This will prevent growth re-spawns. You might have to replace it now and then if it gets destroyed in a raid.