You need the swamp key to open the crypts that you find.

It sounds like you need to find another swamp biome if you've run out of crypts in this one. As I said above to go exploring you would be wise to build a Karve at the very least. I'm not able to get in the game to check but from memory you need bronze nails, fine wood, resin and deer hide to build the Karve.