You can try clicking (on/in steam) View>Servers>LAN tab to see if it shows up in there. I saw mine listed in there in the past when I was setting this up.

I use "connect to IP" on the Valheim menu when I run a home LAN server.
you would type in something like...

192.168.1.121:2456

...that's mine, but it should look similar, just your local IP # with ":2456" at end.