No. Look for small stone square structures with green torches outside of them. Have an iron gate on each one your find you need to open with that swamp key.

Normally it's the bigger swamps you find crypts in. Although on rare occasions I have found some in smaller swamps.