Trolls also can RNG spawn in place of the Greydwarf groups that harass you out in the world. (I had 2 spawn in two different location that Greydwarves did early in the game, no where near a troll cave.)

So RNG is becoming a major part of the game. As you can just randomly encounter trolls in the Black Forest at any time. I don't know if there is a limit to the number that can spawn into the world... or how close. My experience is they can spawn pretty close.