I ran with troll armor until I got iron armor for the same reason. Entry-level bronze armor is only as good as upgraded troll armor, and you have the movement penalty to deal with. And the next level up doesn't seem worth the difference.

However, armor progression is not linear in Valheim. A few points of extra armor can make a huge difference due to the opaque armor/damage system. If you can stand the grind of mining copper and tin, then getting high-tier bronze armor will surely make your early swamp experience easier. But it's not necessary: you can play carefully with troll armor in the swamp until you have gathered enough iron to craft yourself a set.

Funny enough throughout the entire game I kept a set of troll armor around for whenever I wanted to make a run gathering raspberries, mushrooms, blueberries and thistles (OH GOD THE THISTLES) in the woods. With Eikthyr, it feels you're running on wind!