Every starred mob has different colors. Sometimes hard to see, I give you that. But if a mob attacks me, I IMMEDIATELY try to look at and see the stars. Their healthbar with the stars is hovering above their heads.

Then I can adjust my tactics. Most of the time this works absolutely fine. Very rarely I don't see them right away due to some angle I'm looking at. But once I shift around I usually see them.

Edit: "One eyed swamp man" is a draugr elite. And yes, they can come with stars as well. Had once a two star draugr elite on my tail. Man, that was NOT fun.