There is one quirk to this - you cannot connect to a server, even a dedicated server hosted in your local network, if you don't have a WAN connection to the internet (probably to Steam).

I found this out the hard way when my wife and I wanted to play during a winter storm that knocked out our internet for a week. We had power, our home network was up, I even had a dedicated server running on an old rack server I keep around... we couldn't join it or directly join games hosted by each other. It just timed out.