Draugr archers are a bit over tuned and the main reason to avoid the swamp surface. Dive into the catacombs first thing and start mining iron. Underground is many MANY times safer than the surface.

The jump from black forest to swamp is the single biggest difficulty spike in Valheim. The enemy stat growth is huge, and the constant wet debuff makes avoiding combat more difficult. Everything after it gets easier, at least going in the intended progression order.