Some mods don't need the server to have the mods, such at textures, timers, and stuff that don't change any stats, just visuals. I use a colorblind mod and a clock mod on non modded servers for instance no issues.