I think it can matter, especially in regards to durability. You can bring down any foe with enough skill and patience, but if you're using something truly ineffective, you'll find your weapon breaking after just a few kills. Some enemies can get up to 75% resistant to certain damage types (even immune in some cases), or up to double damage taken, so bringing the right weapons can make a large difference.