Things to check / try:

1) If it's working everywhere else and not in Valheim, make sure you didn't accidentally disable it in Valheims Steam properties. Right click on the game or click the gear cog in the library page and open the properties. There's a checkbox that allows you to enable / disable the overlay for specific games.

2) I haven't checked this and I'm reasonably sure it shouldn't matter these days (it used to) but if you're launching with Vulkan, try using DirectX instead and vice versa.