Swamp is the first (and I think the only) true challenge in Valheim. It's not that hard when you get used to it, but first steps are always the hardest.
Get the best food you can, max your gear, avoid the fight as much as you can and never stay at night. Stealth is the best option, and the knife will be your best friend into crypts. Also always carry lots of potions and a portal to come back to safety faster.
Just keep trying, and if it still too difficult, just ask for some help. Valheim community is always ready to help :)