Do you use mods? I play vanilla and haven't had any trouble finding mountain caves in new areas. So, either you explored closer to those mountains than you think, or you have mods interfering--or you have a borked world and may need to restart (though I think there is something that will let you reset the biomes, but it's in the mod category so I don't bother with it).