That's a good question. Let me check.

I went in to the game, chose 'start server' and 'community server,' and it ran. Whether or not it would allow a connection from another computer on a LAN I cannot test at this time (I only have 1 Valheim here), and I have literally disconnected it from the network to test. If you routed all internet traffic through a specific machine, and disabled the internet while still having your system neighborhood in place, it might possibly work. Those additional machines would also have to be connected offline to Steam, of course. If you have a couple of accounts, you might follow up and then post your findings.

If you're thinking of setting up a LAN during a camping trip or something, I can see looking in to this, but otherwise, it seems like a lot of hassle.