Don't worry- it's not really intuitive. Lots of people have gotten thrown off by this.
I always have my primary weapon in slot #1, my shield in #2, my bow in #3, my ax in #4, and my hammer in #5. If I'm going into a fight, I press 1 and 2. If I am going to shoot something, I press 3. (Make sure to have the correct arrows selected by right-clicking them in your inventory.) If I'm going to chop wood, I press 4, and if I am building I press 5. Hope this helps.