This is my experience as well considering that I have found only 1-3 cores per infested dungeon. So far I have only found them near the coast of my biomes because they are the easiest to see due to the large staircase that leads to them.

The only exception is that I remember falling down a staircase in one of the marble structures located inland that led into a another infested dungeon opening. The unfortunate thing is that I died at the entrance before I could mark it on my map and I simply cannot find it again.