This should be easy with Thunderstore mod manager, because it allows to run modded or vanilla game without any mod deleting/disabling/etc.

To be honest the best option is that everyone has the same mods. With Thunderstore it's easy, because you just export mod list key, give it to your friends, they import keys and download your mod list.

But other solutions like Vortex might be problematic. For example we play with difficulty mods (enemies up to 5 stars). Due to some problem with Vortex my mod got disabled. The effect wast that enemies spawned, they had up to 5 stars, but I was able to see only 2 stars - 3 or more stars were displayed as no star. I died in 15 minutes killed by 3 or 4 star zombie in 2 hits because I've ignored him as no star mob.