Instead of on the shortcut link, do the '-window-mode exclusive' via the Steam Library - go and find Valheim and right click and under Properties find Launch options and enter '-window-mode exclusive' there and it should work much better for you.

Also, if in game, just do an Alt-Enter and back again if you want to confirm your fps is where you expected it to be or if you want to double check. Hope this fixes your issue!