From my experience the wisp torches do not prevent mob spawns, mine also always get destroyed, but the mats are still there so they can be replaced easily, but its kind of annoying.