You can select the item in inventory and press the action button (pretty sure it's the action button) on keyboard as well as just dragging it out with the mouse. I imagine the same on controller, select the item and press whatever the action button is?