I finally found the swamp. It is actually to the right of the island where The Elder is located. Crazy dangerous place. But I'm glad I have found it. I have also set up a Portal right between the swamp and The Elder.