I faced the same problem and I fixed it about 5 minutes ago. There is how I solved it;

Right click on Valheim in your Library and choose Properties.

In General, there is a tab named Launch Options. Write “ -windows-mode exclusive -screen-fullscreen “ this here and close the Properties.

Launch the game. This should be helpful, I hope.