When I tried this at the beginning of the year (internet service was down due to a major snowstorm, but our home LAN was working fine), it was unable to connect. Nothing had changed in our server / client config since it worked last, only our internet service had been cut. My understanding is that if the clients and server can't handshake through the SteamWorks API and Steam Servers, multiplayer can't work, even if you have clients and a server all on the same LAN that work just fine.

@Mandos good question, I don't know. but i bet it works.
in the installation directory of the Valheim Dedicated Server is a start_headless_server.bat file to start the server, run it. and don't close the cmd window, otherwise the server will shut down.

@bansheedud17, when you said "Offline LAN" I assumed you were all on the same router.

You can do a LAN when you're all on the same local network. You just use the IP assigned by the router as opposed to your ISP assigned external IP.

As for the game starting offline I think it is possible yes but out of curiosity I'll give it a try later.

Is that something we type in manually or can we all just connect to the same router?

well, once u type ure Ip in and do ure password , as long as u dont do a nother ip or seed it should stay there when u go to ure server , but note i would write donwn that ip just in case theres a update that will erase it like the time they did with the crossover patch

all my ips and seeds where gone from that , luckly i write all my worlds down , i think u might be able to find them in ure files in valheim , that im not sure on though, someone with better info could tell ya

Does the server even start when it can't reach Steam though?

Yes. if you're hosting a server, on your PC. Each of your friends connects via directIP.
via your LANip, for example 192.168.178.20