Killing Bonemass activates 'You are being hunted'' I believe only killing him on another world will prevent the event, though I don't have firsthand experience of having done that.

There are actually only four wolves but they will continue to respawn until the timer runs out.