The "You are being hunted" mob is unique in that it requires you to be AWAY from your base (or specifically, three crafting stations that trigger other mobs). Like everyone has said, it triggers after you've killed Bonemass on THAT world. Alji is right, only four wolves at any one time but they'll keep coming for the full two minutes as long as you keep killing them. As has been noted this is a fantastic source of delicious wolf meat. If you're low on health and have no food (shame on you) when the event triggers, try to get as high as possible on a terrain feature where they can't reach you. They WILL destroy any player made structures to get to you. I'm running in end-game fully upgraded padded armor and they're no threat since they can only be zero-star wolves. Sword, Porcupine, or Flesh Rippers will do the trick.