I don't find them that hard to kill with bronze/iron armor and an iron sword. Fire arrows work too, but that takes forever. A bit of healing potion is handy in case they hit you.

I just with they wouldn't spawn as often. It's annoying to spend half your time in the swamp fighting them.