I forget how to get it to appear on the server list, but I know you should be able to connect directly using IP ;port. I use a gportal server now, but my friend originally set it up on a spare box using this guide :

https://www.pcgamer.com/valheim-multiplayer-dedicated-server/