Well what I did was I built a house on stilts, and then raised the ground up under the house and my land, and I built a dock for my boat, they can swim all they want, they wont get in, occasionally I head one banging on my door on the fence line, but its not that big of an issue.

My problem is now I can't really extend my land much, I got friends I play with and Im in the middle of them so I can only build up my land so far.

My other buddy next to me dug out around his base, close to water level, the land is like on a side of a hill, he built a wall around his land on the low side and then put spikes between the wall and the land up top, so they sometimes fall into the hole and kill themselfs, works pretty good. The water side he lined up spikes down the coast on his property, he has a dock and a boat, sometimes we hear one of them swimming around his boat, but have been easy to just go a shoot them with a bow and go on doing what we were doing.

Its not hard, multiple ways of avoiding the swimmers, But they sure do like the boats so I would recommend not having land across the water from you home, we had Trolls come by and beat the crap out of our boats from the land cross from us during a night raid.