Does your server satisfy minimum requirements? Are you running the client on it, too? That would increase the requirements. Have you made sure that the game is hosted from dedicated?