Wiki says XP is typically per valid action. So any swing counts as long as it hits a hostile, chopping has to hit wood, bows have to hit targets(not birds), every block, etc. Any XP gained over time counts fractions of XP, so sprinting/swimming/riding(sprint only) gives XP.