Right click like you're blocking. That will let you pivot in the direction you want. Open the inventory after you are facing the right way. Moving with the inventory open doesn't seem to reset the direction.

Added: but your character will still throw it. Hard to hit exactly the spot you want.