if they play on your modded server then they can have the same mods but not the other way around. the server is the one to control mod use or no mods. presonally, i hate valheim vanilla. it's too grindy which takes away the fun factor for me. i have my own modded server and it's the only way i can play valheim. never going back to vanilla.