At night, greydwarves can spawn near your base, as do skeletons, and fulings depending on your current boss kill level, in the meadows, and else where.