I had the some problem. Something that worked for me was going to Steam -> Settings -> In-Game -> Server Browser Pings / Minutes and lowering the value to 250. Might be worth a shot.