As M.Red pointed out, mark on your map all the purple-flamed Greydwarf Nests, and either take them out (smash the nest to bits, it will drop an Ancient Seed, necessary to spawn the 2nd boss), or give them a very wide berth, their detection/agro range is quite large, and they will continually spawn and attack you in never ending waves.