Make sure to sleep right before the fight, at night time the dwarves will spawn a lot more