Certain skills will use less stamina as you level them up. Swimming, running, I think the weapons themselves, etc.