Well, IMHO, you should skill up Clubs. Clubs are great against almost everything. Plus the 2 best weapons in the game, Frostner, and Porcupine are clubs. Also, Axes is slower than clubs, I think.
If you are trying to save space, then yeah having one less tool/weapon is an option.
Using an axe for combat and for woodcutting trees are 2 different skill sets, they train seprately
If you like swords, thats cool, train up swords. Once I get my clubs to 70, Ill probably do the sword next, have a L4 Green metal sword waiting
Remember to keep a big boulder in your home or in your camp with roof and fire so you can practice your combat skills all night long to train them up.