@Mandos good question, I don't know. but i bet it works.
in the installation directory of the Valheim Dedicated Server is a start_headless_server.bat file to start the server, run it. and don't close the cmd window, otherwise the server will shut down.

@bansheedud17, when you said "Offline LAN" I assumed you were all on the same router.