I have turned off valheim because I am afraid that my graphics card will overheat.

There is a FPS limiter available in the settings, you might want to try that one. What GPU do you have? If it's an older one, it might need a good cleaning and new thermal paste. That did help with my aging GTX 1070Ti that reached prior to that temperatures of over 90 deg. Celsius.

And yes, the main menu is particularily taxing on GPUs.