Everything that is explored, and yes, it also counts if you just've been near it, even if it is NOT explored by you, counts as explored.

So, if you wish to get the new content, as with every expansion, you need to re-roll your map. You might just create a new map with the same seed, so you already know the parts you explored.

That's what I did when the Ice Caves were released. But then I was just in Bronze Age and didn't really have much.