Valheim uses the Steam controller API, so if it works in Steam it should work in Valheim.

That said, I use a bluetooth XBox controller, and I've had issues that sound similar, although not as bad, as @rexpiscator. My controller occasionally does not work in Valheim when I start it. Fortunately I just have to power off / on the controller to get it to work.

I'm running Windows 11.