The 4 Pillars that surround his summoning altar cannot be destroyed. I just hide behind those and shoot him with a Finewood Bow and Fire Arrows. When he spawns the roots, just move to the opposite side. The pillars will protect you from his vines that he shoots out.

Maybe have one of you shooting him while the other runs protection from Greydwarves. And make sure you do it during the daytime, mob spawns are much greater during the night.