On top of what GunsForBucks has said, some servers require specific mods that all its players must have. Eg: most server-side Valheim+ hosts also requires its players to have it too.

Your best bet is to start your own, or find an existing group.