If you encounter a ghost, you visit the Swamp at night.

Which isn't the best time to do that.

What you could try is to wait outside the area until dawn, make sure you got a bow and arrows equipped and as soon as it's light, sail in slowly.
While doing so, keep changing your point of view, so you will "map" the area around you. This will give you a clue if / where any leeches are present.

When you've spotted those, release the rudder, pick up the bow and shoot the leeches.
Yes, you will drop out on loot by doing that, but at least you're able to clear the path. Goes the same with the skellies and draugr; though as they walk over the bottom of the water, it can be harder.

Progress slowly towards your tombstone(s) and beyond that to make sure the entire area is cleared out.
Only then return to your tombstone and retrieve your items, store what isn't needed aboard your ship's hold and sail away.

A second method is to find a dungeon as close as possible to where you died, build a teleporter there or nearby and use that to quickly access and leave the swamp area.
Again, make sure you clear out the area before actually progressing.

Thorin :)