I usually see slots tied to RAM. Your dedicated server hosted only 4 people which makes me think it had very little in the way of RAM which will cause a lot of lag, especially when more people are present.

I'm just going off the information you've given. I don't know what your server actually offered beyond that.