I did a whole bunch of caves for a Fenris set (plus upgrades) on the run I'm currently on.

Some things I noticed:

1) Caves are RNG and rare. Not every mountain has a cave.
2) Little weenie mountains never have caves. They seem to have the same height requirement as drakes.
3) They can be hard to spot but it's generally easier from above and below. I like to run around the whole mountain at the base looking up and inward, then climb up and look outward and down. I try to make sure I check the entire mountain at least twice from different angles.
4) It's easier to see the big mountains from the water. You can see the nice, big mountains from a long ways away but once you land your boat your view can be blocked by trees and terrain.
5) Don't be afraid to run through plains. You can kill deathsquitoes in swamp gear / food easily as long as they don't catch you completely flat-footed, avoid fulings and lox.
6) Parry stone golems with an upgraded silver shield then smash them with an iron mace. They're actually pretty easy once you get the hang of it. Don't use Frostner against them.