There are some performance tweaks tips out there, which might help improve the performance.

https://www.pcgamer.com/valheim-increase-fps-performance/
https://steamcommunity.com/sharedfiles/filedetails/?id=2390029674

Maybe it also helps to use vsync to cap the FPS

I remember i had some performance problems until i tried those tweaks. (Linux, RTX3060)