I copied both, "AppData\LocalLow\IronGate\Valheim" and "\steamapps\common\Valheim". Tried out the mistlands and had some fun.
Decided to go back and play with friends. Deleted the folders, told Steam I'm not playing Beta anymore and renamed the backup copies back. Managed to restore the character with Steam Cloud. Lost the world.

Until they implement a tool that converts your saves between versions, I'd advise to stay out of non-default versions.