Yeah the biggest mobs make the loudest noise, which draws the most trash...

I failed to an unexpected golem on my first attempt too, and took a break from bosses for awhile after that. Eventually I went back, and wound up inadvertently getting boss aggro at the very base of the mountain. Probably slid down by mistake, but Moder never broke off.
At that point, I didn't seem to need to worry about mountain trash adds; I was on the border of black forest. Iirc, after that it was just arrows. I don't think I tried meleeing. I had no silver gear since I didn't know silver existed at the time.

So you might try kiting her to a safer area to fight?