I can confirm what OP is saying, I saw 3 skellies in the middle of my meadows biome as well near the water. And no there's no black forest in the vicinity. I've survived for 106 days so far and this is the first time I suddenly saw them so close to my base.

There was no attack, they did not follow me because I rarely even go to the black forest anymore and when I do it's through a portal to a whole other island.