Rename:
E:\SteamLibrary\steamapps\common\Valheim\doorstop_config.ini to .bak
(or whever yours is)
Now BepInEx won't get loaded, this no mods loaded.

When you want to go back, rename it back.