probably something bugged, I'm slightly faster RX 6600 XT, playing in Vulkan and I'm using additionally a mod to extend render distances (https://www.nexusmods.com/valheim/mods/1842) and I'm having around 100fps at max in-game settings with exception active points light shodows (because I have ton of various lights in based and I don't need each of them to cast individual shadows as it's stresses system quite a bit and adds visually very little)

Try switching to Vulkan and see if that helps. Add these launch parameters:

-window-mode exclusive -screen-fullscreen -force-vulkan