lodbias [0-5] – Set draw distance for server.

This is a command for the server that may help. Howto use here:
https://www.ginx.tv/en/valheim/all-valheim-server-admin-commands-how-to-use

Also it is early access so its not abnormal to have issues:
https://dathost.net/blog/2021/03/18/valheim-common-performance-issues-and-solutions/