Some tactics
They can't destroy the big trees. Kite them around places they will have to go around to get to you that you can pass through.

They don't seem to go too far into other biomes. Pull them out and fight them after they turn around and run away.

I have started about 20 worlds with 20 new characters. So I understand the situation.

After I killed one or two I just ran away from them, I was in troll armor. I was just scouting for iron crypts anyway. They don't seem to bother destroying a portal on top of a crypt. So you can mine around them easily. As long as you are fed and rested you can outrun them/lose them fairly easily. Once in iron you can tank them.