Do note they seem to be in high aggression mode and won't lose you unless you get out of load distance or more likely they get distracted by the other mobs. They're annoying for this reason, no amount of sneak will help you avoid them, like an event. They despawn once day is up though. (though a bug was present when H&H was first out they wouldn't, same with Fenrings and Wraiths, but this was fixed).