If your friend is set to Offline, IIRC you can't join them via the "Join Game/Lobby" option.

All I ever do is go by Join Game on my Steam friend's list.