Most will require the other players to have the mods installed, else it could cause problems.

From what I've been told, things like calculations are determined using the host, the example I was given was if you had a mod that gave you the ability to fire a fireball at an enemy, only the host with the mod will see the fireball, the other players would not see the fireball, however the damage would still count towards the enemy's health. This can lead to either the experience being awkward, or might even cause actual problems and crashes.

Someone will have to correct me if I got something wrong, however I will stand by the idea that everyone installing the mods themselves as well as the host is the best option.