A DGX-1 has quite a lot of power. However, when using it I only utilize 34% of one of eight cards. Are there some points and tweaks I might missed? I know I can't parallelize everything, there is an upper limit. But to be honest, I expected a better performance.
I'm running a nvidia-docker with tensorflow preinstalled. The script running is from dennybritz which is quite well used. I ran the docker one one card. Since it didn't utilized the whole card I didn't gave him second one. Would this have any benefit? Of course I could ran multiple instances and pick the best one. But I'd rather have results sooner when having 170TFLOPS accessible.