I think it's unity and the shadows/lights system. As I said, I had the exact same problems with other unity games who wasn't optimized until they were and the changes are massive.