By default the dedicated server uses the world named "Dedicated", if it doesn't exist it will be created. You can see this by pressing F2 (left upper edge -> World) while playing

To change that you have to modify the file "start_headless_server.bat" in your Steam folder (....\steamapps\common\Valheim dedicated server) .. or , if you're using STEAMCMD for dedicated, where you've installed it

To be more specific, change the "-world" parameter in the last line from '-world "Dedicated"' to '-world "MyFancyOtherWorld"' or the name of your world you want to play.

The dedicated server uses the same folder for worlds as your regular local game. And here comes possible trouble - this increases the risk of loosing your character and world files by steam-cloud-sync errors. So BACKUP your char and world files (before and regularly after doing this) - turning off steam-cloud-sync might also be a good idea

Your char and world files are located at "C:\Users\insertusername\AppData\LocalLow\IronGate\Valheim" (insert your username)

edit: there should be a file "Valheim Dedicated Server Manual.pdf" in the dedicated server's directory which might contain aditional helpfull information