Please look here (https://valheim.fandom.com/wiki/Greydwarf) - there are conditions of spawning dwarves. Maybe you was in the event (red circle on the mini map) ? After it creatures remain and you must deal with them ...