I don't think you can toggle off just the minimap alone but there's a key to hide the UI. I think it's F3. You won't see your quick access item bar though, IIRC.